2026-07-18 10:27:59,637 [ 709262 ] INFO : ClickHouse root is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse (runner:42, check_args_and_update_paths) 2026-07-18 10:27:59,637 [ 709262 ] INFO : Cases dir is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration (runner:86, check_args_and_update_paths) 2026-07-18 10:27:59,637 [ 709262 ] INFO : utils dir is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse/utils (runner:97, check_args_and_update_paths) 2026-07-18 10:27:59,637 [ 709262 ] INFO : base_configs_dir: /home/ubuntu/_work/ClickHouse/ClickHouse/programs/server, binary: /home/ubuntu/_work/_temp/test/build/clickhouse, cases_dir: /home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration (runner:99, check_args_and_update_paths) clickhouse_integration_tests_volume Running pytest container as: 'docker run --rm --name clickhouse_integration_tests_kxmknw --privileged --dns-search='.' --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-odbc-bridge:/clickhouse-odbc-bridge --volume=/home/ubuntu/_work/_temp/test/build/clickhouse:/clickhouse --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-library-bridge:/clickhouse-library-bridge --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/programs/server:/clickhouse-config --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration:/ClickHouse/tests/integration --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/backupview:/ClickHouse/utils/backupview --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/grpc-client/pb2:/ClickHouse/utils/grpc-client/pb2 --volume=/run:/run/host:ro --volume=clickhouse_integration_tests_volume:/var/lib/docker -e DOCKER_DOTNET_CLIENT_TAG=11de0b29a15d -e DOCKER_HELPER_TAG=2cffe1eae894 -e DOCKER_BASE_TAG=b74d853ebe76 -e DOCKER_KERBEROS_KDC_TAG=9391ecdee8d7 -e DOCKER_MYSQL_GOLANG_CLIENT_TAG=9bec2a638e6e -e DOCKER_MYSQL_JAVA_CLIENT_TAG=766bff31cfe4 -e DOCKER_MYSQL_JS_CLIENT_TAG=41ba7c2ec2a1 -e DOCKER_MYSQL_PHP_CLIENT_TAG=88be89c1e3b6 -e DOCKER_NGINX_DAV_TAG=b55ac9cd7519 -e DOCKER_POSTGRESQL_JAVA_CLIENT_TAG=a4eff5c7f4d6 -e DOCKER_PYTHON_BOTTLE_TAG=caad4729259e -e DOCKER_CLIENT_TIMEOUT=300 -e COMPOSE_HTTP_TIMEOUT=600 -e PYTHONUNBUFFERED=1 -e PYTEST_ADDOPTS=" -rfEps --run-id=1 --color=no --durations=0 'test_storage_s3_queue/test.py::test_shards_distributed[ordered-8]' -vvv" altinityinfra/integration-tests-runner:cf95077f4609 '. Start tests ============================= test session starts ============================== platform linux -- Python 3.10.12, pytest-7.4.4, pluggy-1.5.0 -- /usr/bin/python3 cachedir: .pytest_cache rootdir: /ClickHouse/tests/integration configfile: pytest.ini plugins: xdist-3.5.0, random-0.2, order-1.0.1, repeat-0.9.4, reportlog-0.4.0, timeout-2.2.0 timeout: 900.0s timeout method: signal timeout func_only: False collecting ... collected 1 item test_storage_s3_queue/test.py::test_shards_distributed[ordered-8] FAILED [100%] =================================== FAILURES =================================== ______________________ test_shards_distributed[ordered-8] ______________________ started_cluster = mode = 'ordered', processing_threads = 8 @pytest.mark.parametrize( "mode, processing_threads", [ pytest.param("unordered", 1), pytest.param("unordered", 8), pytest.param("ordered", 1), pytest.param("ordered", 8), ], ) def test_shards_distributed(started_cluster, mode, processing_threads): node = started_cluster.instances["instance"] node_2 = started_cluster.instances["instance2"] table_name = f"test_shards_distributed_{mode}_{processing_threads}" dst_table_name = f"{table_name}_dst" keeper_path = f"/clickhouse/test_{table_name}" files_path = f"{table_name}_data" files_to_generate = 300 row_num = 50 total_rows = row_num * files_to_generate shards_num = 2 i = 0 for instance in [node, node_2]: create_table( started_cluster, instance, table_name, mode, files_path, additional_settings={ "keeper_path": keeper_path, "s3queue_processing_threads_num": processing_threads, "s3queue_buckets": shards_num, }, ) i += 1 for instance in [node, node_2]: create_mv(instance, table_name, dst_table_name) total_values = generate_random_files( started_cluster, files_path, files_to_generate, row_num=row_num ) def get_count(node, table_name): return int(run_query(node, f"SELECT count() FROM {table_name}")) def print_debug_info(): processed_files = ( node.query( f""" select splitByChar('/', file_name)[-1] as file from system.s3queue where zookeeper_path ilike '%{table_name}%' and status = 'Processed' and rows_processed > 0 order by file """ ) .strip() .split("\n") ) logging.debug( f"Processed files by node 1: {len(processed_files)}/{files_to_generate}" ) processed_files = ( node_2.query( f""" select splitByChar('/', file_name)[-1] as file from system.s3queue where zookeeper_path ilike '%{table_name}%' and status = 'Processed' and rows_processed > 0 order by file """ ) .strip() .split("\n") ) logging.debug( f"Processed files by node 2: {len(processed_files)}/{files_to_generate}" ) count = get_count(node, dst_table_name) + get_count(node_2, dst_table_name) logging.debug(f"Processed rows: {count}/{total_rows}") info = node.query( f""" select concat('test_', toString(number), '.csv') as file from numbers(300) where file not in (select splitByChar('/', file_name)[-1] from clusterAllReplicas(default, system.s3queue) where zookeeper_path ilike '%{table_name}%' and status = 'Processed' and rows_processed > 0) """ ) logging.debug(f"Unprocessed files: {info}") files1 = ( node.query( f""" select splitByChar('/', file_name)[-1] from system.s3queue where zookeeper_path ilike '%{table_name}%' and status = 'Processed' and rows_processed > 0 """ ) .strip() .split("\n") ) files2 = ( node_2.query( f""" select splitByChar('/', file_name)[-1] from system.s3queue where zookeeper_path ilike '%{table_name}%' and status = 'Processed' and rows_processed > 0 """ ) .strip() .split("\n") ) def intersection(list_a, list_b): return [e for e in list_a if e in list_b] logging.debug(f"Intersecting files: {intersection(files1, files2)}") for _ in range(30): if ( get_count(node, dst_table_name) + get_count(node_2, dst_table_name) ) == total_rows: break time.sleep(1) if ( get_count(node, dst_table_name) + get_count(node_2, dst_table_name) ) != total_rows: print_debug_info() assert False get_query = f"SELECT column1, column2, column3 FROM {dst_table_name}" res1 = [list(map(int, l.split())) for l in run_query(node, get_query).splitlines()] res2 = [ list(map(int, l.split())) for l in run_query(node_2, get_query).splitlines() ] if len(res1) + len(res2) != total_rows or len(res1) <= 0 or len(res2) <= 0 or True: logging.debug( f"res1 size: {len(res1)}, res2 size: {len(res2)}, total_rows: {total_rows}" ) print_debug_info() assert len(res1) + len(res2) == total_rows # Checking that all engines have made progress assert len(res1) > 0 > assert len(res2) > 0 E assert 0 > 0 E + where 0 = len([]) test_storage_s3_queue/test.py:1468: AssertionError ---------------------------- Captured stdout setup ----------------------------- Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml ------------------------------ Captured log setup ------------------------------ 2026-07-18 08:28:02 [ 514 ] DEBUG : Command:['docker ps | wc -l'] (cluster.py:113, run_and_check) 2026-07-18 08:28:02 [ 514 ] DEBUG : Stdout:1 (cluster.py:121, run_and_check) 2026-07-18 08:28:02 [ 514 ] DEBUG : No running containers (conftest.py:95, cleanup_environment) 2026-07-18 08:28:02 [ 514 ] DEBUG : Pruning Docker networks (conftest.py:97, cleanup_environment) 2026-07-18 08:28:02 [ 514 ] DEBUG : Command:['docker network prune --force'] (cluster.py:113, run_and_check) 2026-07-18 08:28:02 [ 514 ] DEBUG : Command:["sysctl net.ipv4.ip_local_port_range='55000 65535'"] (cluster.py:113, run_and_check) 2026-07-18 08:28:02 [ 514 ] DEBUG : Stdout:net.ipv4.ip_local_port_range = 55000 65535 (cluster.py:121, run_and_check) 2026-07-18 08:28:02 [ 514 ] DEBUG : ENV DOCKER_KERBEROS_KDC_TAG 9391ecdee8d7 (cluster.py:446, __init__) 2026-07-18 08:28:02 [ 514 ] DEBUG : ENV CLICKHOUSE_TESTS_SERVER_BIN_PATH /clickhouse (cluster.py:446, __init__) 2026-07-18 08:28:02 [ 514 ] DEBUG : ENV MSAN_OPTIONS abort_on_error=1 poison_in_dtor=1 (cluster.py:446, __init__) 2026-07-18 08:28:02 [ 514 ] DEBUG : ENV JAVA_TOOL_OPTIONS -Djdk.attach.allowAttachSelf=true (cluster.py:446, __init__) 2026-07-18 08:28:02 [ 514 ] DEBUG : ENV TSAN_OPTIONS halt_on_error=1 abort_on_error=1 history_size=7 memory_limit_mb=46080 second_deadlock_stack=1 (cluster.py:446, __init__) 2026-07-18 08:28:02 [ 514 ] DEBUG : ENV HOSTNAME 319ac98b1633 (cluster.py:446, __init__) 2026-07-18 08:28:02 [ 514 ] DEBUG : ENV SHLVL 0 (cluster.py:446, __init__) 2026-07-18 08:28:02 [ 514 ] DEBUG : ENV HOME /root (cluster.py:446, __init__) 2026-07-18 08:28:02 [ 514 ] DEBUG : ENV OLDPWD / (cluster.py:446, __init__) 2026-07-18 08:28:02 [ 514 ] DEBUG : ENV DOCKER_HELPER_TAG 2cffe1eae894 (cluster.py:446, __init__) 2026-07-18 08:28:02 [ 514 ] DEBUG : ENV PYTHONUNBUFFERED 1 (cluster.py:446, __init__) 2026-07-18 08:28:02 [ 514 ] DEBUG : ENV DOCKER_PYTHON_BOTTLE_TAG caad4729259e (cluster.py:446, __init__) 2026-07-18 08:28:02 [ 514 ] DEBUG : ENV UBSAN_OPTIONS print_stacktrace=1 (cluster.py:446, __init__) 2026-07-18 08:28:02 [ 514 ] DEBUG : ENV PYTEST_ADDOPTS -rfEps --run-id=1 --color=no --durations=0 'test_storage_s3_queue/test.py::test_shards_distributed[ordered-8]' -vvv (cluster.py:446, __init__) 2026-07-18 08:28:02 [ 514 ] DEBUG : ENV CLICKHOUSE_LIBRARY_BRIDGE_BINARY_PATH /clickhouse-library-bridge (cluster.py:446, __init__) 2026-07-18 08:28:02 [ 514 ] DEBUG : ENV COMPOSE_HTTP_TIMEOUT 600 (cluster.py:446, __init__) 2026-07-18 08:28:02 [ 514 ] DEBUG : ENV DOCKER_MYSQL_PHP_CLIENT_TAG 88be89c1e3b6 (cluster.py:446, __init__) 2026-07-18 08:28:02 [ 514 ] DEBUG : ENV DOCKER_DOTNET_CLIENT_TAG 11de0b29a15d (cluster.py:446, __init__) 2026-07-18 08:28:02 [ 514 ] DEBUG : ENV CLICKHOUSE_TESTS_CLIENT_BIN_PATH /clickhouse (cluster.py:446, __init__) 2026-07-18 08:28:02 [ 514 ] DEBUG : ENV DOCKER_MYSQL_JS_CLIENT_TAG 41ba7c2ec2a1 (cluster.py:446, __init__) 2026-07-18 08:28:02 [ 514 ] DEBUG : ENV PATH /spark-3.3.2-bin-hadoop3/bin:/opt/gdb/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin (cluster.py:446, __init__) 2026-07-18 08:28:02 [ 514 ] DEBUG : ENV DOCKER_KERBERIZED_HADOOP_TAG latest (cluster.py:446, __init__) 2026-07-18 08:28:02 [ 514 ] DEBUG : ENV DOCKER_CHANNEL stable (cluster.py:446, __init__) 2026-07-18 08:28:02 [ 514 ] DEBUG : ENV DOCKER_CLIENT_TIMEOUT 300 (cluster.py:446, __init__) 2026-07-18 08:28:02 [ 514 ] DEBUG : ENV DOCKER_POSTGRESQL_JAVA_CLIENT_TAG a4eff5c7f4d6 (cluster.py:446, __init__) 2026-07-18 08:28:02 [ 514 ] DEBUG : ENV DOCKER_NGINX_DAV_TAG b55ac9cd7519 (cluster.py:446, __init__) 2026-07-18 08:28:02 [ 514 ] DEBUG : ENV DOCKER_MYSQL_GOLANG_CLIENT_TAG 9bec2a638e6e (cluster.py:446, __init__) 2026-07-18 08:28:02 [ 514 ] DEBUG : ENV PWD /ClickHouse/tests/integration (cluster.py:446, __init__) 2026-07-18 08:28:02 [ 514 ] DEBUG : ENV DOCKER_MYSQL_JAVA_CLIENT_TAG 766bff31cfe4 (cluster.py:446, __init__) 2026-07-18 08:28:02 [ 514 ] DEBUG : ENV CLICKHOUSE_ODBC_BRIDGE_BINARY_PATH /clickhouse-odbc-bridge (cluster.py:446, __init__) 2026-07-18 08:28:02 [ 514 ] DEBUG : ENV CLICKHOUSE_TESTS_BASE_CONFIG_DIR /clickhouse-config (cluster.py:446, __init__) 2026-07-18 08:28:02 [ 514 ] DEBUG : ENV TZ Etc/UTC (cluster.py:446, __init__) 2026-07-18 08:28:02 [ 514 ] DEBUG : ENV JAVA_PATH /usr/lib/jvm/java-11-openjdk-amd64/bin/java (cluster.py:446, __init__) 2026-07-18 08:28:02 [ 514 ] DEBUG : ENV DOCKER_BASE_TAG b74d853ebe76 (cluster.py:446, __init__) 2026-07-18 08:28:02 [ 514 ] DEBUG : ENV SPARK_HOME /spark-3.3.2-bin-hadoop3 (cluster.py:446, __init__) 2026-07-18 08:28:02 [ 514 ] DEBUG : ENV LC_CTYPE C.UTF-8 (cluster.py:446, __init__) 2026-07-18 08:28:02 [ 514 ] DEBUG : ENV INTEGRATION_TESTS_RUN_ID 1 (cluster.py:446, __init__) 2026-07-18 08:28:02 [ 514 ] DEBUG : ENV WORKER_FREE_PORTS 30000 30001 30002 30003 30004 30005 30006 30007 30008 30009 30010 30011 30012 30013 30014 30015 30016 30017 30018 30019 30020 30021 30022 30023 30024 30025 30026 30027 30028 30029 30030 30031 30032 30033 30034 30035 30036 30037 30038 30039 30040 30041 30042 30043 30044 30045 30046 30047 30048 30049 (cluster.py:446, __init__) 2026-07-18 08:28:02 [ 514 ] DEBUG : ENV PYTEST_CURRENT_TEST test_storage_s3_queue/test.py::test_shards_distributed[ordered-8] (setup) (cluster.py:446, __init__) 2026-07-18 08:28:02 [ 514 ] DEBUG : CLUSTER INIT base_config_dir:/clickhouse-config (cluster.py:766, __init__) 2026-07-18 08:28:02 [ 514 ] DEBUG : clickhouse_start_command: clickhouse server --config-file=/etc/clickhouse-server/{main_config_file} --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log (cluster.py:1787, add_instance) 2026-07-18 08:28:02 [ 514 ] DEBUG : Setup Keeper (cluster.py:1059, setup_keeper_cmd) 2026-07-18 08:28:02 [ 514 ] DEBUG : Cluster name: project_name:rootteststorages3queue. Added instance name:instance tag:b74d853ebe76 base_cmd:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/.env', '--project-name', 'rootteststorages3queue', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml'] docker_compose_yml_dir:/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/ (cluster.py:2081, add_instance) 2026-07-18 08:28:02 [ 514 ] DEBUG : clickhouse_start_command: clickhouse server --config-file=/etc/clickhouse-server/{main_config_file} --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log (cluster.py:1787, add_instance) 2026-07-18 08:28:02 [ 514 ] DEBUG : Cluster name: project_name:rootteststorages3queue. Added instance name:instance2 tag:b74d853ebe76 base_cmd:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/.env', '--project-name', 'rootteststorages3queue', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance2/docker-compose.yml'] docker_compose_yml_dir:/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/ (cluster.py:2081, add_instance) 2026-07-18 08:28:02 [ 514 ] DEBUG : clickhouse_start_command: clickhouse server --config-file=/etc/clickhouse-server/{main_config_file} --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log (cluster.py:1787, add_instance) 2026-07-18 08:28:02 [ 514 ] DEBUG : Cluster name: project_name:rootteststorages3queue. Added instance name:old_instance tag:23.12 base_cmd:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/.env', '--project-name', 'rootteststorages3queue', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance2/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/old_instance/docker-compose.yml'] docker_compose_yml_dir:/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/ (cluster.py:2081, add_instance) 2026-07-18 08:28:02 [ 514 ] DEBUG : clickhouse_start_command: clickhouse server --config-file=/etc/clickhouse-server/{main_config_file} --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log (cluster.py:1787, add_instance) 2026-07-18 08:28:02 [ 514 ] DEBUG : Cluster name: project_name:rootteststorages3queue. Added instance name:instance_too_many_parts tag:b74d853ebe76 base_cmd:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/.env', '--project-name', 'rootteststorages3queue', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance2/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/old_instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance_too_many_parts/docker-compose.yml'] docker_compose_yml_dir:/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/ (cluster.py:2081, add_instance) 2026-07-18 08:28:02 [ 514 ] DEBUG : clickhouse_start_command: clickhouse server --config-file=/etc/clickhouse-server/{main_config_file} --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log (cluster.py:1787, add_instance) 2026-07-18 08:28:02 [ 514 ] DEBUG : Cluster name: project_name:rootteststorages3queue. Added instance name:instance_24.5 tag:24.5 base_cmd:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/.env', '--project-name', 'rootteststorages3queue', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance2/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/old_instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance_too_many_parts/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance_24.5/docker-compose.yml'] docker_compose_yml_dir:/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/ (cluster.py:2081, add_instance) 2026-07-18 08:28:02 [ 514 ] INFO : Starting cluster... (test.py:143, started_cluster) 2026-07-18 08:28:02 [ 514 ] INFO : Running tests in /ClickHouse/tests/integration/test_storage_s3_queue/test.py (cluster.py:2790, start) 2026-07-18 08:28:02 [ 514 ] DEBUG : Cluster start called. is_up=False (cluster.py:2797, start) 2026-07-18 08:28:02 [ 514 ] DEBUG : Docker networks for project rootteststorages3queue are NETWORK ID NAME DRIVER SCOPE (cluster.py:855, print_all_docker_pieces) 2026-07-18 08:28:02 [ 514 ] DEBUG : Docker containers for project rootteststorages3queue are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:863, print_all_docker_pieces) 2026-07-18 08:28:02 [ 514 ] DEBUG : Docker volumes for project rootteststorages3queue are DRIVER VOLUME NAME (cluster.py:871, print_all_docker_pieces) 2026-07-18 08:28:02 [ 514 ] DEBUG : Cleanup called (cluster.py:876, cleanup) 2026-07-18 08:28:02 [ 514 ] DEBUG : Docker networks for project rootteststorages3queue are NETWORK ID NAME DRIVER SCOPE (cluster.py:855, print_all_docker_pieces) 2026-07-18 08:28:02 [ 514 ] DEBUG : Docker containers for project rootteststorages3queue are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:863, print_all_docker_pieces) 2026-07-18 08:28:02 [ 514 ] DEBUG : Docker volumes for project rootteststorages3queue are DRIVER VOLUME NAME (cluster.py:871, print_all_docker_pieces) 2026-07-18 08:28:02 [ 514 ] DEBUG : Command:docker container list --all --filter name='^/rootteststorages3queue_.*_1$' --format '{{.ID}}:{{.Names}}' (cluster.py:113, run_and_check) 2026-07-18 08:28:02 [ 514 ] DEBUG : Unstopped containers: {} (cluster.py:890, cleanup) 2026-07-18 08:28:02 [ 514 ] DEBUG : No running containers for project: rootteststorages3queue (cluster.py:904, cleanup) 2026-07-18 08:28:02 [ 514 ] DEBUG : Trying to prune unused networks... (cluster.py:910, cleanup) 2026-07-18 08:28:02 [ 514 ] DEBUG : Trying to prune unused volumes... (cluster.py:937, cleanup) 2026-07-18 08:28:02 [ 514 ] DEBUG : Command:['docker volume ls | wc -l'] (cluster.py:113, run_and_check) 2026-07-18 08:28:02 [ 514 ] DEBUG : Stdout:1 (cluster.py:121, run_and_check) 2026-07-18 08:28:02 [ 514 ] DEBUG : Setup directory for instance: instance (cluster.py:2810, start) 2026-07-18 08:28:02 [ 514 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4538, create_dir) 2026-07-18 08:28:02 [ 514 ] DEBUG : Create directory for common tests configuration (cluster.py:4543, create_dir) 2026-07-18 08:28:02 [ 514 ] DEBUG : Copy common configuration from helpers (cluster.py:4563, create_dir) 2026-07-18 08:28:02 [ 514 ] DEBUG : Generate and write macros file (cluster.py:4606, create_dir) 2026-07-18 08:28:02 [ 514 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_storage_s3_queue/configs/zookeeper.xml', '/ClickHouse/tests/integration/test_storage_s3_queue/configs/s3queue_log.xml'] to /ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance/configs/config.d (cluster.py:4636, create_dir) 2026-07-18 08:28:02 [ 514 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance/database (cluster.py:4653, create_dir) 2026-07-18 08:28:02 [ 514 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance/logs (cluster.py:4664, create_dir) 2026-07-18 08:28:02 [ 514 ] DEBUG : Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon; coproc tail -f /dev/null; wait $$!" (cluster.py:4750, create_dir) 2026-07-18 08:28:02 [ 514 ] DEBUG : Setup directory for instance: instance2 (cluster.py:2810, start) 2026-07-18 08:28:02 [ 514 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4538, create_dir) 2026-07-18 08:28:02 [ 514 ] DEBUG : Create directory for common tests configuration (cluster.py:4543, create_dir) 2026-07-18 08:28:02 [ 514 ] DEBUG : Copy common configuration from helpers (cluster.py:4563, create_dir) 2026-07-18 08:28:02 [ 514 ] DEBUG : Generate and write macros file (cluster.py:4606, create_dir) 2026-07-18 08:28:02 [ 514 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_storage_s3_queue/configs/s3queue_log.xml'] to /ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance2/configs/config.d (cluster.py:4636, create_dir) 2026-07-18 08:28:02 [ 514 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance2/database (cluster.py:4653, create_dir) 2026-07-18 08:28:02 [ 514 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance2/logs (cluster.py:4664, create_dir) 2026-07-18 08:28:02 [ 514 ] DEBUG : Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon; coproc tail -f /dev/null; wait $$!" (cluster.py:4750, create_dir) 2026-07-18 08:28:02 [ 514 ] DEBUG : Setup directory for instance: old_instance (cluster.py:2810, start) 2026-07-18 08:28:02 [ 514 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4538, create_dir) 2026-07-18 08:28:02 [ 514 ] DEBUG : Create directory for common tests configuration (cluster.py:4543, create_dir) 2026-07-18 08:28:02 [ 514 ] DEBUG : Copy common configuration from helpers (cluster.py:4563, create_dir) 2026-07-18 08:28:02 [ 514 ] DEBUG : Generate and write macros file (cluster.py:4606, create_dir) 2026-07-18 08:28:02 [ 514 ] DEBUG : Copy custom test config files [] to /ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/old_instance/configs/config.d (cluster.py:4636, create_dir) 2026-07-18 08:28:02 [ 514 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/old_instance/database (cluster.py:4653, create_dir) 2026-07-18 08:28:02 [ 514 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/old_instance/logs (cluster.py:4664, create_dir) 2026-07-18 08:28:02 [ 514 ] DEBUG : Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon; coproc tail -f /dev/null; wait $$!" (cluster.py:4750, create_dir) 2026-07-18 08:28:02 [ 514 ] DEBUG : Setup directory for instance: instance_too_many_parts (cluster.py:2810, start) 2026-07-18 08:28:02 [ 514 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4538, create_dir) 2026-07-18 08:28:02 [ 514 ] DEBUG : Create directory for common tests configuration (cluster.py:4543, create_dir) 2026-07-18 08:28:02 [ 514 ] DEBUG : Copy common configuration from helpers (cluster.py:4563, create_dir) 2026-07-18 08:28:02 [ 514 ] DEBUG : Generate and write macros file (cluster.py:4606, create_dir) 2026-07-18 08:28:02 [ 514 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_storage_s3_queue/configs/s3queue_log.xml', '/ClickHouse/tests/integration/test_storage_s3_queue/configs/merge_tree.xml'] to /ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance_too_many_parts/configs/config.d (cluster.py:4636, create_dir) 2026-07-18 08:28:02 [ 514 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance_too_many_parts/database (cluster.py:4653, create_dir) 2026-07-18 08:28:02 [ 514 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance_too_many_parts/logs (cluster.py:4664, create_dir) 2026-07-18 08:28:02 [ 514 ] DEBUG : Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon; coproc tail -f /dev/null; wait $$!" (cluster.py:4750, create_dir) 2026-07-18 08:28:02 [ 514 ] DEBUG : Setup directory for instance: instance_24.5 (cluster.py:2810, start) 2026-07-18 08:28:02 [ 514 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4538, create_dir) 2026-07-18 08:28:02 [ 514 ] DEBUG : Create directory for common tests configuration (cluster.py:4543, create_dir) 2026-07-18 08:28:02 [ 514 ] DEBUG : Copy common configuration from helpers (cluster.py:4563, create_dir) 2026-07-18 08:28:02 [ 514 ] DEBUG : Generate and write macros file (cluster.py:4606, create_dir) 2026-07-18 08:28:02 [ 514 ] DEBUG : Copy custom test config files [] to /ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance_24.5/configs/config.d (cluster.py:4636, create_dir) 2026-07-18 08:28:02 [ 514 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance_24.5/database (cluster.py:4653, create_dir) 2026-07-18 08:28:02 [ 514 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance_24.5/logs (cluster.py:4664, create_dir) 2026-07-18 08:28:02 [ 514 ] DEBUG : Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon; coproc tail -f /dev/null; wait $$!" (cluster.py:4750, create_dir) 2026-07-18 08:28:02 [ 514 ] DEBUG : Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw', 'keeper_binary': '/clickhouse', 'keeper_cmd_prefix': 'clickhouse keeper', 'image': 'altinityinfra/integration-test:b74d853ebe76', 'user': '0', 'keeper_fs': 'bind', 'keeper_logs_dir1': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper1/log', 'keeper_config_dir1': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper1/config', 'keeper_db_dir1': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper1/coordination', 'keeper_logs_dir2': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper2/log', 'keeper_config_dir2': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper2/config', 'keeper_db_dir2': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper2/coordination', 'keeper_logs_dir3': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper3/log', 'keeper_config_dir3': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper3/config', 'keeper_db_dir3': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper3/coordination', 'MINIO_CERTS_DIR': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/minio/certs', 'MINIO_DATA_DIR': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/minio/data', 'MINIO_PORT': '9001', 'SSL_CERT_FILE': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/minio/certs/public.crt', 'AZURITE_PORT': '30000', 'AZURITE_STORAGE_ACCOUNT_URL': 'http://azurite1:30000/devstoreaccount1', 'AZURITE_CONNECTION_STRING': 'DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://azurite1:30000/devstoreaccount1;'} stored in /ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/.env (cluster.py:86, _create_env_file) 2026-07-18 08:28:02 [ 514 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file) 2026-07-18 08:28:02 [ 514 ] DEBUG : No config file found (config.py:28, find_config_file) 2026-07-18 08:28:02 [ 514 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file) 2026-07-18 08:28:02 [ 514 ] DEBUG : No config file found (config.py:28, find_config_file) 2026-07-18 08:28:02 [ 514 ] DEBUG : http://localhost:None "GET /version HTTP/1.1" 200 825 (connectionpool.py:547, _make_request) 2026-07-18 08:28:02 [ 514 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/.env', '--project-name', 'rootteststorages3queue', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance2/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/old_instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance_too_many_parts/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance_24.5/docker-compose.yml', 'pull'] (cluster.py:113, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling proxy1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling zoo1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling proxy2 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling resolver ... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling zoo2 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling minio1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling azurite1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling zoo3 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling instance_too_many_parts ... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling instance ... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling instance_24.5 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling old_instance ... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling instance2 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling minio1 ... pulling from minio/minio (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling minio1 ... digest: sha256:9eea83a4e1425067e7... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling minio1 ... status: image is up to date for m... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling minio1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling zoo2 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling zoo2 ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling zoo2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling zoo2 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling instance_24.5 ... pulling from clickhouse/clickhous... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling azurite1 ... pulling from altinityinfra/azurite (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling zoo3 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling azurite1 ... digest: sha256:dae2a5f96553962901... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling azurite1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling instance_24.5 ... digest: sha256:f1155aa4adec32964c... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling instance_24.5 ... status: image is up to date for c... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling azurite1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling instance_24.5 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling zoo3 ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling zoo3 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling zoo3 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling instance_too_many_parts ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling instance_too_many_parts ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling instance_too_many_parts ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling instance_too_many_parts ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling old_instance ... pulling from clickhouse/clickhous... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling old_instance ... digest: sha256:5d0802b64c09f6d2c6... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling old_instance ... status: image is up to date for c... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling old_instance ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling resolver ... pulling from altinityinfra/python... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling resolver ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling resolver ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling zoo1 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling zoo1 ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling zoo1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling zoo1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling proxy1 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling proxy1 ... digest: sha256:92e86c3de6b8ff00d8... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling proxy1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling proxy1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling instance2 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling proxy2 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling instance2 ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling instance2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling instance2 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling proxy2 ... digest: sha256:92e86c3de6b8ff00d8... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling proxy2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling proxy2 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling instance ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling instance ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling instance ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Stderr:Pulling instance ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:04 [ 514 ] DEBUG : Setup ZooKeeper (cluster.py:2853, start) 2026-07-18 08:28:04 [ 514 ] DEBUG : Creating internal ZooKeeper dirs: ['/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper1/log', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper1/config', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper1/coordination', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper2/log', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper2/config', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper2/coordination', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper3/log', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper3/config', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper3/coordination'] (cluster.py:2854, start) 2026-07-18 08:28:04 [ 514 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/.env', '--project-name', 'rootteststorages3queue', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--verbose', 'up', '-d'] (cluster.py:113, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.config.config.find: Using configuration files: /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.docker_client.get_client: docker-compose version 1.29.2, build unknown (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:docker-py version: (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:CPython version: 3.10.12 (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:OpenSSL version: OpenSSL 3.0.2 15 Mar 2022 (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.docker_client.get_client: Docker base_url: http+docker://localhost (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.docker_client.get_client: Docker version: Platform={'Name': 'Docker Engine - Community'}, Components=[{'Name': 'Engine', 'Version': '23.0.6', 'Details': {'ApiVersion': '1.42', 'Arch': 'amd64', 'BuildTime': '2023-05-05T21:18:13.000000000+00:00', 'Experimental': 'false', 'GitCommit': '9dbdbd4', 'GoVersion': 'go1.19.9', 'KernelVersion': '5.15.0-185-generic', 'MinAPIVersion': '1.12', 'Os': 'linux'}}, {'Name': 'containerd', 'Version': 'v2.2.6', 'Details': {'GitCommit': '11ce9d5f3c68c941867e82890e93e815c1304f1b'}}, {'Name': 'runc', 'Version': '1.3.6', 'Details': {'GitCommit': 'v1.3.6-0-g491b69ba'}}, {'Name': 'docker-init', 'Version': '0.19.0', 'Details': {'GitCommit': 'de40ad0'}}], Version=23.0.6, ApiVersion=1.42, MinAPIVersion=1.12, GitCommit=9dbdbd4, GoVersion=go1.19.9, Os=linux, Arch=amd64, KernelVersion=5.15.0-185-generic, BuildTime=2023-05-05T21:18:13.000000000+00:00 (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('rootteststorages3queue_default') (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker info <- () (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker info -> {'Architecture': 'x86_64', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'BridgeNfIp6tables': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'BridgeNfIptables': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'CPUSet': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'CPUShares': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'CgroupDriver': 'cgroupfs', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'CgroupVersion': '2', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'ContainerdCommit': {'Expected': '11ce9d5f3c68c941867e82890e93e815c1304f1b', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'ID': '11ce9d5f3c68c941867e82890e93e815c1304f1b'}, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Containers': 0, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('rootteststorages3queue_default') (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.network.ensure: Creating network "rootteststorages3queue_default" with the default driver (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_network <- (name='rootteststorages3queue_default', driver=None, options=None, ipam=None, internal=False, enable_ipv6=False, labels={'com.docker.compose.project': 'rootteststorages3queue', 'com.docker.compose.network': 'default', 'com.docker.compose.version': '1.29.2'}, attachable=True, check_duplicate=True) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_network -> {'Id': '68edf820af72704e6a13b5765842403851e4df6ba40406dba5e2f4bce112ffa6', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Warning': ''} (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:b74d853ebe76') (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:b74d853ebe76') (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:b74d853ebe76') (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, , } (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_zoo1_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='rootteststorages3queue', service='zoo1', number=1)} (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='rootteststorages3queue', service='zoo1', number=1) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_zoo3_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='rootteststorages3queue', service='zoo3', number=1)} (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='rootteststorages3queue', service='zoo3', number=1) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:b74d853ebe76') (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_zoo2_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='rootteststorages3queue', service='zoo2', number=1)} (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='rootteststorages3queue', service='zoo2', number=1) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:b74d853ebe76') (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:b74d853ebe76') (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:b74d853ebe76') (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: e1b002f5f7163cd43f00f79d36e32d4aecbf6ae231db5857377f23464ff8a8b7 (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={}, binds=[], volumes_from=[], privileged=False, network_mode='rootteststorages3queue_default', devices=None, device_requests=None, dns=None, dns_opt=['attempts:2', 'timeout:1', 'inet6', 'rotate'], dns_search=None, restart_policy={'Name': 'always', 'MaximumRetryCount': 0}, runtime=None, cap_add=['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=['label:disable'], ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=[{'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper3/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper3/config', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper3/log', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}], device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': [], (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Links': [], (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Mounts': [{'ReadOnly': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper3/coordination', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Target': '/var/lib/clickhouse-keeper', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Type': 'bind'}, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: {'ReadOnly': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (entrypoint='clickhouse keeper --config=/etc/clickhouse-keeper/keeper_config3.xml --log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log --errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log', image='altinityinfra/integration-test:b74d853ebe76', user='0', volumes={}, name='rootteststorages3queue_zoo3_1', detach=True, environment=[], labels={'com.docker.compose.project': 'rootteststorages3queue', 'com.docker.compose.service': 'zoo3', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/ClickHouse/tests/integration/compose', 'com.docker.compose.project.config_files': '/ClickHouse/tests/integration/compose/docker_compose_keeper.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': 'e1b002f5f7163cd43f00f79d36e32d4aecbf6ae231db5857377f23464ff8a8b7'}, host_config={'NetworkMode': 'rootteststorages3queue_default', 'RestartPolicy': {'Name': 'always', 'MaximumRetryCount': 0}, 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], 'SecurityOpt': ['label:disable'], 'VolumesFrom': [], 'Binds': [], 'PortBindings': {}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}, 'Mounts': [{'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper3/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper3/config', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper3/log', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}]}, networking_config={'EndpointsConfig': {'rootteststorages3queue_default': {'Aliases': ['zoo3'], 'IPAMConfig': {}}}}) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:b74d853ebe76') (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:b74d853ebe76') (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: 7400babad73fc5b6b4378bbcbf6361e30c12b5fd2784b9f74cc14b9488ebb555 (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={}, binds=[], volumes_from=[], privileged=False, network_mode='rootteststorages3queue_default', devices=None, device_requests=None, dns=None, dns_opt=['attempts:2', 'timeout:1', 'inet6', 'rotate'], dns_search=None, restart_policy={'Name': 'always', 'MaximumRetryCount': 0}, runtime=None, cap_add=['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=['label:disable'], ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=[{'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper2/config', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper2/log', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper2/coordination', 'Type': 'bind', 'ReadOnly': None}], device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': [], (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Links': [], (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Mounts': [{'ReadOnly': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper2/config', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Target': '/etc/clickhouse-keeper', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Type': 'bind'}, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: {'ReadOnly': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (entrypoint='clickhouse keeper --config=/etc/clickhouse-keeper/keeper_config2.xml --log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log --errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log', image='altinityinfra/integration-test:b74d853ebe76', user='0', volumes={}, name='rootteststorages3queue_zoo2_1', detach=True, environment=[], labels={'com.docker.compose.project': 'rootteststorages3queue', 'com.docker.compose.service': 'zoo2', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/ClickHouse/tests/integration/compose', 'com.docker.compose.project.config_files': '/ClickHouse/tests/integration/compose/docker_compose_keeper.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': '7400babad73fc5b6b4378bbcbf6361e30c12b5fd2784b9f74cc14b9488ebb555'}, host_config={'NetworkMode': 'rootteststorages3queue_default', 'RestartPolicy': {'Name': 'always', 'MaximumRetryCount': 0}, 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], 'SecurityOpt': ['label:disable'], 'VolumesFrom': [], 'Binds': [], 'PortBindings': {}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}, 'Mounts': [{'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper2/config', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper2/log', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper2/coordination', 'Type': 'bind', 'ReadOnly': None}]}, networking_config={'EndpointsConfig': {'rootteststorages3queue_default': {'Aliases': ['zoo2'], 'IPAMConfig': {}}}}) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: b49500cf00515390296e3a78f952e69b566049b259ffe8721adcdfd687c29b8e (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={}, binds=[], volumes_from=[], privileged=False, network_mode='rootteststorages3queue_default', devices=None, device_requests=None, dns=None, dns_opt=['attempts:2', 'timeout:1', 'inet6', 'rotate'], dns_search=None, restart_policy={'Name': 'always', 'MaximumRetryCount': 0}, runtime=None, cap_add=['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=['label:disable'], ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=[{'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper1/config', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper1/log', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}], device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': [], (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Links': [], (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Mounts': [{'ReadOnly': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper1/config', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Target': '/etc/clickhouse-keeper', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Type': 'bind'}, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: {'ReadOnly': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (entrypoint='clickhouse keeper --config=/etc/clickhouse-keeper/keeper_config1.xml --log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log --errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log', image='altinityinfra/integration-test:b74d853ebe76', user='0', volumes={}, name='rootteststorages3queue_zoo1_1', detach=True, environment=[], labels={'com.docker.compose.project': 'rootteststorages3queue', 'com.docker.compose.service': 'zoo1', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/ClickHouse/tests/integration/compose', 'com.docker.compose.project.config_files': '/ClickHouse/tests/integration/compose/docker_compose_keeper.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': 'b49500cf00515390296e3a78f952e69b566049b259ffe8721adcdfd687c29b8e'}, host_config={'NetworkMode': 'rootteststorages3queue_default', 'RestartPolicy': {'Name': 'always', 'MaximumRetryCount': 0}, 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], 'SecurityOpt': ['label:disable'], 'VolumesFrom': [], 'Binds': [], 'PortBindings': {}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}, 'Mounts': [{'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper1/config', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper1/log', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}]}, networking_config={'EndpointsConfig': {'rootteststorages3queue_default': {'Aliases': ['zoo1'], 'IPAMConfig': {}}}}) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': '629e59ca2bd58378e382b49b054100ea9106be98ddd11a6d2bd1ffbe1fcde657', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Warnings': []} (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('629e59ca2bd58378e382b49b054100ea9106be98ddd11a6d2bd1ffbe1fcde657') (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': '5852136cd9392dd79f7ee8d6320819eaf5cd5e2b033fb675c937845cb486c533', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Warnings': []} (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('5852136cd9392dd79f7ee8d6320819eaf5cd5e2b033fb675c937845cb486c533') (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': '732ce5e607697aa126a740328b276089d0d2ce5a04462823f13ffa579bfc83ec', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Warnings': []} (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('732ce5e607697aa126a740328b276089d0d2ce5a04462823f13ffa579bfc83ec') (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: '--config=/etc/clickhouse-keeper/keeper_config3.xml', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Cmd': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('629e59ca2bd58378e382b49b054100ea9106be98ddd11a6d2bd1ffbe1fcde657', 'rootteststorages3queue_default') (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: '--config=/etc/clickhouse-keeper/keeper_config2.xml', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Cmd': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('732ce5e607697aa126a740328b276089d0d2ce5a04462823f13ffa579bfc83ec', 'rootteststorages3queue_default') (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: '--config=/etc/clickhouse-keeper/keeper_config1.xml', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Cmd': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('5852136cd9392dd79f7ee8d6320819eaf5cd5e2b033fb675c937845cb486c533', 'rootteststorages3queue_default') (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('629e59ca2bd58378e382b49b054100ea9106be98ddd11a6d2bd1ffbe1fcde657', 'rootteststorages3queue_default', aliases=['629e59ca2bd5', 'zoo3'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('732ce5e607697aa126a740328b276089d0d2ce5a04462823f13ffa579bfc83ec', 'rootteststorages3queue_default', aliases=['zoo2', '732ce5e60769'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('5852136cd9392dd79f7ee8d6320819eaf5cd5e2b033fb675c937845cb486c533', 'rootteststorages3queue_default', aliases=['zoo1', '5852136cd939'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('629e59ca2bd58378e382b49b054100ea9106be98ddd11a6d2bd1ffbe1fcde657') (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('732ce5e607697aa126a740328b276089d0d2ce5a04462823f13ffa579bfc83ec') (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('5852136cd9392dd79f7ee8d6320819eaf5cd5e2b033fb675c937845cb486c533') (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='rootteststorages3queue', service='zoo1', number=1) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_zoo1_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='rootteststorages3queue', service='zoo3', number=1) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_zoo3_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='rootteststorages3queue', service='zoo2', number=1) (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_zoo2_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:05 [ 514 ] DEBUG : Wait ZooKeeper to start (cluster.py:2506, wait_zookeeper_to_start) 2026-07-18 08:28:05 [ 514 ] DEBUG : get_instance_ip instance_name=zoo1 (cluster.py:2137, get_instance_ip) 2026-07-18 08:28:05 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/rootteststorages3queue_zoo1_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:28:05 [ 514 ] DEBUG : get_kazoo_client: zoo1, ip:172.16.1.3, port:2181, use_ssl:False (cluster.py:3290, get_kazoo_client) 2026-07-18 08:28:05 [ 514 ] INFO : Connecting to 172.16.1.3(172.16.1.3):2181, use_ssl: False (connection.py:650, _connect) 2026-07-18 08:28:05 [ 514 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt) 2026-07-18 08:28:06 [ 514 ] INFO : Connecting to 172.16.1.3(172.16.1.3):2181, use_ssl: False (connection.py:650, _connect) 2026-07-18 08:28:06 [ 514 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt) 2026-07-18 08:28:06 [ 514 ] INFO : Connecting to 172.16.1.3(172.16.1.3):2181, use_ssl: False (connection.py:650, _connect) 2026-07-18 08:28:06 [ 514 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt) 2026-07-18 08:28:06 [ 514 ] INFO : Connecting to 172.16.1.3(172.16.1.3):2181, use_ssl: False (connection.py:650, _connect) 2026-07-18 08:28:06 [ 514 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt) 2026-07-18 08:28:07 [ 514 ] INFO : Connecting to 172.16.1.3(172.16.1.3):2181, use_ssl: False (connection.py:650, _connect) 2026-07-18 08:28:07 [ 514 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt) 2026-07-18 08:28:08 [ 514 ] INFO : Connecting to 172.16.1.3(172.16.1.3):2181, use_ssl: False (connection.py:650, _connect) 2026-07-18 08:28:08 [ 514 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt) 2026-07-18 08:28:10 [ 514 ] INFO : Connecting to 172.16.1.3(172.16.1.3):2181, use_ssl: False (connection.py:650, _connect) 2026-07-18 08:28:10 [ 514 ] DEBUG : Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=10000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) (connection.py:312, _submit) 2026-07-18 08:28:10 [ 514 ] INFO : Zookeeper connection established, state: CONNECTED (client.py:532, _session_callback) 2026-07-18 08:28:10 [ 514 ] DEBUG : Sending request(xid=1): GetChildren(path='/', watcher=None) (connection.py:312, _submit) 2026-07-18 08:28:10 [ 514 ] DEBUG : Received response(xid=1): ['keeper'] (connection.py:410, _read_response) 2026-07-18 08:28:10 [ 514 ] DEBUG : Sending request(xid=2): Close() (connection.py:312, _submit) 2026-07-18 08:28:10 [ 514 ] WARNING : Connection dropped: socket connection broken (connection.py:622, _connect_attempt) 2026-07-18 08:28:10 [ 514 ] WARNING : Transition to CONNECTING (connection.py:626, _connect_attempt) 2026-07-18 08:28:10 [ 514 ] INFO : Zookeeper connection lost (client.py:543, _session_callback) 2026-07-18 08:28:10 [ 514 ] WARNING : Failed connecting to Zookeeper within the connection retry policy. (connection.py:515, zk_loop) 2026-07-18 08:28:10 [ 514 ] INFO : Zookeeper session closed, state: CLOSED (client.py:537, _session_callback) 2026-07-18 08:28:10 [ 514 ] DEBUG : get_instance_ip instance_name=zoo2 (cluster.py:2137, get_instance_ip) 2026-07-18 08:28:10 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/rootteststorages3queue_zoo2_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:28:10 [ 514 ] DEBUG : get_kazoo_client: zoo2, ip:172.16.1.4, port:2181, use_ssl:False (cluster.py:3290, get_kazoo_client) 2026-07-18 08:28:10 [ 514 ] INFO : Connecting to 172.16.1.4(172.16.1.4):2181, use_ssl: False (connection.py:650, _connect) 2026-07-18 08:28:10 [ 514 ] DEBUG : Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=10000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) (connection.py:312, _submit) 2026-07-18 08:28:10 [ 514 ] INFO : Zookeeper connection established, state: CONNECTED (client.py:532, _session_callback) 2026-07-18 08:28:10 [ 514 ] DEBUG : Sending request(xid=1): GetChildren(path='/', watcher=None) (connection.py:312, _submit) 2026-07-18 08:28:10 [ 514 ] DEBUG : Received response(xid=1): ['keeper'] (connection.py:410, _read_response) 2026-07-18 08:28:10 [ 514 ] DEBUG : Sending request(xid=2): Close() (connection.py:312, _submit) 2026-07-18 08:28:10 [ 514 ] WARNING : Connection dropped: socket connection broken (connection.py:622, _connect_attempt) 2026-07-18 08:28:10 [ 514 ] WARNING : Transition to CONNECTING (connection.py:626, _connect_attempt) 2026-07-18 08:28:10 [ 514 ] INFO : Zookeeper connection lost (client.py:543, _session_callback) 2026-07-18 08:28:10 [ 514 ] WARNING : Failed connecting to Zookeeper within the connection retry policy. (connection.py:515, zk_loop) 2026-07-18 08:28:10 [ 514 ] INFO : Zookeeper session closed, state: CLOSED (client.py:537, _session_callback) 2026-07-18 08:28:10 [ 514 ] DEBUG : get_instance_ip instance_name=zoo3 (cluster.py:2137, get_instance_ip) 2026-07-18 08:28:10 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/rootteststorages3queue_zoo3_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:28:10 [ 514 ] DEBUG : get_kazoo_client: zoo3, ip:172.16.1.2, port:2181, use_ssl:False (cluster.py:3290, get_kazoo_client) 2026-07-18 08:28:10 [ 514 ] INFO : Connecting to 172.16.1.2(172.16.1.2):2181, use_ssl: False (connection.py:650, _connect) 2026-07-18 08:28:10 [ 514 ] DEBUG : Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=10000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) (connection.py:312, _submit) 2026-07-18 08:28:10 [ 514 ] INFO : Zookeeper connection established, state: CONNECTED (client.py:532, _session_callback) 2026-07-18 08:28:10 [ 514 ] DEBUG : Sending request(xid=1): GetChildren(path='/', watcher=None) (connection.py:312, _submit) 2026-07-18 08:28:10 [ 514 ] DEBUG : Received response(xid=1): ['keeper'] (connection.py:410, _read_response) 2026-07-18 08:28:10 [ 514 ] DEBUG : Sending request(xid=2): Close() (connection.py:312, _submit) 2026-07-18 08:28:10 [ 514 ] WARNING : Connection dropped: socket connection broken (connection.py:622, _connect_attempt) 2026-07-18 08:28:10 [ 514 ] WARNING : Transition to CONNECTING (connection.py:626, _connect_attempt) 2026-07-18 08:28:10 [ 514 ] INFO : Zookeeper connection lost (client.py:543, _session_callback) 2026-07-18 08:28:11 [ 514 ] WARNING : Failed connecting to Zookeeper within the connection retry policy. (connection.py:515, zk_loop) 2026-07-18 08:28:11 [ 514 ] INFO : Zookeeper session closed, state: CLOSED (client.py:537, _session_callback) 2026-07-18 08:28:11 [ 514 ] DEBUG : All instances of ZooKeeper Secure started (cluster.py:2521, wait_zookeeper_nodes_to_start) 2026-07-18 08:28:11 [ 514 ] INFO : Trying to create Minio instance by command docker-compose --env-file /ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/.env --project-name rootteststorages3queue --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml --verbose up -d (cluster.py:3091, start) 2026-07-18 08:28:11 [ 514 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/.env', '--project-name', 'rootteststorages3queue', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--verbose', 'up', '-d'] (cluster.py:113, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.config.config.find: Using configuration files: /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.docker_client.get_client: docker-compose version 1.29.2, build unknown (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:docker-py version: (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:CPython version: 3.10.12 (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:OpenSSL version: OpenSSL 3.0.2 15 Mar 2022 (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.docker_client.get_client: Docker base_url: http+docker://localhost (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.docker_client.get_client: Docker version: Platform={'Name': 'Docker Engine - Community'}, Components=[{'Name': 'Engine', 'Version': '23.0.6', 'Details': {'ApiVersion': '1.42', 'Arch': 'amd64', 'BuildTime': '2023-05-05T21:18:13.000000000+00:00', 'Experimental': 'false', 'GitCommit': '9dbdbd4', 'GoVersion': 'go1.19.9', 'KernelVersion': '5.15.0-185-generic', 'MinAPIVersion': '1.12', 'Os': 'linux'}}, {'Name': 'containerd', 'Version': 'v2.2.6', 'Details': {'GitCommit': '11ce9d5f3c68c941867e82890e93e815c1304f1b'}}, {'Name': 'runc', 'Version': '1.3.6', 'Details': {'GitCommit': 'v1.3.6-0-g491b69ba'}}, {'Name': 'docker-init', 'Version': '0.19.0', 'Details': {'GitCommit': 'de40ad0'}}], Version=23.0.6, ApiVersion=1.42, MinAPIVersion=1.12, GitCommit=9dbdbd4, GoVersion=go1.19.9, Os=linux, Arch=amd64, KernelVersion=5.15.0-185-generic, BuildTime=2023-05-05T21:18:13.000000000+00:00 (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('rootteststorages3queue_default') (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network -> {'Attachable': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'ConfigFrom': {'Network': ''}, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'ConfigOnly': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Containers': {'5852136cd9392dd79f7ee8d6320819eaf5cd5e2b033fb675c937845cb486c533': {'EndpointID': '474d421d202ff1eeaf9fa1c48e758ea19e9cc8b915b331742c7deaee36351542', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'IPv4Address': '172.16.1.3/24', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'IPv6Address': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'MacAddress': '02:42:ac:10:01:03', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Name': 'rootteststorages3queue_zoo1_1'}, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: '629e59ca2bd58378e382b49b054100ea9106be98ddd11a6d2bd1ffbe1fcde657': {'EndpointID': '81fd0ab76340b488be48d614dbfc21409b9bd7add1b2a8cb536cd04a051ea91f', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'IPv4Address': '172.16.1.2/24', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_volume <- ('rootteststorages3queue_data1-1') (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker info <- () (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker info -> {'Architecture': 'x86_64', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'BridgeNfIp6tables': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'BridgeNfIptables': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'CPUSet': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'CPUShares': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'CgroupDriver': 'cgroupfs', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'CgroupVersion': '2', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'ContainerdCommit': {'Expected': '11ce9d5f3c68c941867e82890e93e815c1304f1b', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'ID': '11ce9d5f3c68c941867e82890e93e815c1304f1b'}, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Containers': 3, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('rootteststorages3queue_default') (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network -> {'Attachable': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'ConfigFrom': {'Network': ''}, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'ConfigOnly': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Containers': {'5852136cd9392dd79f7ee8d6320819eaf5cd5e2b033fb675c937845cb486c533': {'EndpointID': '474d421d202ff1eeaf9fa1c48e758ea19e9cc8b915b331742c7deaee36351542', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'IPv4Address': '172.16.1.3/24', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'IPv6Address': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'MacAddress': '02:42:ac:10:01:03', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Name': 'rootteststorages3queue_zoo1_1'}, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: '629e59ca2bd58378e382b49b054100ea9106be98ddd11a6d2bd1ffbe1fcde657': {'EndpointID': '81fd0ab76340b488be48d614dbfc21409b9bd7add1b2a8cb536cd04a051ea91f', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'IPv4Address': '172.16.1.2/24', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_volume <- ('rootteststorages3queue_data1-1') (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.volume.initialize: Creating volume "rootteststorages3queue_data1-1" with default driver (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_volume <- ('rootteststorages3queue_data1-1', None, None, labels={'com.docker.compose.project': 'rootteststorages3queue', 'com.docker.compose.volume': 'data1-1', 'com.docker.compose.version': '1.29.2'}) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_volume -> {'CreatedAt': '2026-07-18T08:28:11Z', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Driver': 'local', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Labels': {'com.docker.compose.project': 'rootteststorages3queue', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'com.docker.compose.version': '1.29.2', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'com.docker.compose.volume': 'data1-1'}, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Mountpoint': '/var/lib/docker/volumes/rootteststorages3queue_data1-1/_data', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Name': 'rootteststorages3queue_data1-1', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Options': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Scope': 'local'} (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 3 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 3 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('5852136cd9392dd79f7ee8d6320819eaf5cd5e2b033fb675c937845cb486c533') (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: '--config=/etc/clickhouse-keeper/keeper_config1.xml', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Cmd': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('732ce5e607697aa126a740328b276089d0d2ce5a04462823f13ffa579bfc83ec') (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: '--config=/etc/clickhouse-keeper/keeper_config2.xml', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Cmd': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('629e59ca2bd58378e382b49b054100ea9106be98ddd11a6d2bd1ffbe1fcde657') (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: '--config=/etc/clickhouse-keeper/keeper_config3.xml', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Cmd': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.project.find_orphan_containers: Found orphan containers (rootteststorages3queue_zoo1_1, rootteststorages3queue_zoo2_1, rootteststorages3queue_zoo3_1) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up. (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=proxy1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=proxy1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=proxy2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=proxy2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=minio1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=minio1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=resolver', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=resolver', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/s3-proxy') (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Cmd': ['/run.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Entrypoint': ['/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/s3-proxy') (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Cmd': ['/run.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Entrypoint': ['/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('minio/minio:RELEASE.2024-07-31T05-46-26Z') (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Cmd': ['minio'], (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Entrypoint': ['/usr/bin/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/python-bottle:caad4729259e') (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Cmd': ['python3'], (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=proxy1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=proxy1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=proxy2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=proxy2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.project._get_convergence_plans: minio1 has upstream changes (proxy1, proxy2) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'status': ['created', 'exited'], 'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=minio1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'status': ['created', 'exited'], 'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=minio1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=minio1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=minio1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=minio1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=minio1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.project._get_convergence_plans: resolver has upstream changes (proxy1, proxy2) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'status': ['created', 'exited'], 'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=resolver', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'status': ['created', 'exited'], 'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=resolver', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=resolver', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=resolver', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=resolver', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=resolver', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, , , } (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=proxy2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=proxy1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=proxy2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=proxy1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_proxy2_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='rootteststorages3queue', service='proxy2', number=1)} (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='rootteststorages3queue', service='proxy2', number=1) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_proxy1_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='rootteststorages3queue', service='proxy1', number=1)} (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='rootteststorages3queue', service='proxy1', number=1) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/s3-proxy') (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/s3-proxy') (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Cmd': ['/run.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Entrypoint': ['/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/s3-proxy') (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Cmd': ['/run.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Entrypoint': ['/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/s3-proxy') (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Cmd': ['/run.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Entrypoint': ['/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: b56c6836595b51d9a20583ad32fc4f8944a1765818910b4dd803f05a12e9f3e4 (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={}, binds=[], volumes_from=[], privileged=False, network_mode='rootteststorages3queue_default', devices=None, device_requests=None, dns=None, dns_opt=None, dns_search=None, restart_policy=None, runtime=None, cap_add=None, cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=None, ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=None, device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': [], (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Links': [], (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'NetworkMode': 'rootteststorages3queue_default', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'PortBindings': {}, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'VolumesFrom': []} (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (image='altinityinfra/s3-proxy', volumes={}, name='rootteststorages3queue_proxy2_1', detach=True, ports=['8080', '80', '443'], environment=[], labels={'com.docker.compose.project': 'rootteststorages3queue', 'com.docker.compose.service': 'proxy2', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/ClickHouse/tests/integration/compose', 'com.docker.compose.project.config_files': '/ClickHouse/tests/integration/compose/docker_compose_minio.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': 'b56c6836595b51d9a20583ad32fc4f8944a1765818910b4dd803f05a12e9f3e4'}, host_config={'NetworkMode': 'rootteststorages3queue_default', 'VolumesFrom': [], 'Binds': [], 'PortBindings': {}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}}, networking_config={'EndpointsConfig': {'rootteststorages3queue_default': {'Aliases': ['proxy2'], 'IPAMConfig': {}}}}) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Cmd': ['/run.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Entrypoint': ['/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: b56c6836595b51d9a20583ad32fc4f8944a1765818910b4dd803f05a12e9f3e4 (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={}, binds=[], volumes_from=[], privileged=False, network_mode='rootteststorages3queue_default', devices=None, device_requests=None, dns=None, dns_opt=None, dns_search=None, restart_policy=None, runtime=None, cap_add=None, cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=None, ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=None, device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': [], (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Links': [], (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'NetworkMode': 'rootteststorages3queue_default', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'PortBindings': {}, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'VolumesFrom': []} (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (image='altinityinfra/s3-proxy', volumes={}, name='rootteststorages3queue_proxy1_1', detach=True, ports=['8080', '80', '443'], environment=[], labels={'com.docker.compose.project': 'rootteststorages3queue', 'com.docker.compose.service': 'proxy1', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/ClickHouse/tests/integration/compose', 'com.docker.compose.project.config_files': '/ClickHouse/tests/integration/compose/docker_compose_minio.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': 'b56c6836595b51d9a20583ad32fc4f8944a1765818910b4dd803f05a12e9f3e4'}, host_config={'NetworkMode': 'rootteststorages3queue_default', 'VolumesFrom': [], 'Binds': [], 'PortBindings': {}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}}, networking_config={'EndpointsConfig': {'rootteststorages3queue_default': {'Aliases': ['proxy1'], 'IPAMConfig': {}}}}) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': '41142ac2c4f637cf206c408e54b5d85940af520ec363a99cae85c7b00d0bb765', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Warnings': []} (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('41142ac2c4f637cf206c408e54b5d85940af520ec363a99cae85c7b00d0bb765') (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': 'd14c2cfabc1d0ebd0a98f90504d13e0761310b227a4620de0c83df05cf85a99d', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Warnings': []} (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('d14c2cfabc1d0ebd0a98f90504d13e0761310b227a4620de0c83df05cf85a99d') (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Args': ['/run.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Cmd': ['/run.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Entrypoint': ['/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Env': ['PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'NGINX_VERSION=1.27.5', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('41142ac2c4f637cf206c408e54b5d85940af520ec363a99cae85c7b00d0bb765', 'rootteststorages3queue_default') (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Args': ['/run.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Cmd': ['/run.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Entrypoint': ['/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Env': ['PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'NGINX_VERSION=1.27.5', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('d14c2cfabc1d0ebd0a98f90504d13e0761310b227a4620de0c83df05cf85a99d', 'rootteststorages3queue_default') (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('41142ac2c4f637cf206c408e54b5d85940af520ec363a99cae85c7b00d0bb765', 'rootteststorages3queue_default', aliases=['41142ac2c4f6', 'proxy1'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('d14c2cfabc1d0ebd0a98f90504d13e0761310b227a4620de0c83df05cf85a99d', 'rootteststorages3queue_default', aliases=['d14c2cfabc1d', 'proxy2'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('41142ac2c4f637cf206c408e54b5d85940af520ec363a99cae85c7b00d0bb765') (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('d14c2cfabc1d0ebd0a98f90504d13e0761310b227a4620de0c83df05cf85a99d') (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, } (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, } (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, } (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='rootteststorages3queue', service='proxy1', number=1) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_proxy1_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='rootteststorages3queue', service='proxy2', number=1) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_proxy2_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, } (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, } (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=resolver', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=minio1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=resolver', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=minio1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_minio1_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='rootteststorages3queue', service='minio1', number=1)} (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='rootteststorages3queue', service='minio1', number=1) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('minio/minio:RELEASE.2024-07-31T05-46-26Z') (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_resolver_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='rootteststorages3queue', service='resolver', number=1)} (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='rootteststorages3queue', service='resolver', number=1) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/python-bottle:caad4729259e') (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Cmd': ['minio'], (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Entrypoint': ['/usr/bin/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Cmd': ['python3'], (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('minio/minio:RELEASE.2024-07-31T05-46-26Z') (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/python-bottle:caad4729259e') (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Cmd': ['minio'], (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Entrypoint': ['/usr/bin/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Cmd': ['python3'], (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: b1042e34e5bcce75abbde6a9b79b8d72fa326864cddb467fd42a2951a0146ca2 (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: 6c39b51bdd1ee6641e47438265edd23f4a911da06c476fb341022c357974cf87 (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={}, binds=['rootteststorages3queue_data1-1:/data1:rw', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/minio/certs:/certs:rw'], volumes_from=[], privileged=False, network_mode='rootteststorages3queue_default', devices=None, device_requests=None, dns=None, dns_opt=None, dns_search=None, restart_policy=None, runtime=None, cap_add=None, cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=None, ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=None, device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={}, binds=[], volumes_from=[], privileged=False, network_mode='rootteststorages3queue_default', devices=None, device_requests=None, dns=None, dns_opt=None, dns_search=None, restart_policy=None, runtime=None, cap_add=None, cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=None, ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=None, device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': ['rootteststorages3queue_data1-1:/data1:rw', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/minio/certs:/certs:rw'], (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Links': [], (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'NetworkMode': 'rootteststorages3queue_default', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'PortBindings': {}, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'VolumesFrom': []} (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': [], (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Links': [], (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'NetworkMode': 'rootteststorages3queue_default', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'PortBindings': {}, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'VolumesFrom': []} (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (command='server --console-address 127.0.0.1:19001 --address :9001 --certs-dir /certs /data1-1', environment=['MINIO_ACCESS_KEY=minio', 'MINIO_SECRET_KEY=minio123', 'MINIO_PROMETHEUS_AUTH_TYPE=public'], image='minio/minio:RELEASE.2024-07-31T05-46-26Z', volumes={'/data1': {}, '/certs': {}}, name='rootteststorages3queue_minio1_1', detach=True, ports=['9001'], labels={'com.docker.compose.project': 'rootteststorages3queue', 'com.docker.compose.service': 'minio1', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/ClickHouse/tests/integration/compose', 'com.docker.compose.project.config_files': '/ClickHouse/tests/integration/compose/docker_compose_minio.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': 'b1042e34e5bcce75abbde6a9b79b8d72fa326864cddb467fd42a2951a0146ca2'}, host_config={'NetworkMode': 'rootteststorages3queue_default', 'VolumesFrom': [], 'Binds': ['rootteststorages3queue_data1-1:/data1:rw', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/minio/certs:/certs:rw'], 'PortBindings': {}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}}, networking_config={'EndpointsConfig': {'rootteststorages3queue_default': {'Aliases': ['minio1'], 'IPAMConfig': {}}}}) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (image='altinityinfra/python-bottle:caad4729259e', tty=True, volumes={}, name='rootteststorages3queue_resolver_1', detach=True, ports=['8080'], environment=[], labels={'com.docker.compose.project': 'rootteststorages3queue', 'com.docker.compose.service': 'resolver', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/ClickHouse/tests/integration/compose', 'com.docker.compose.project.config_files': '/ClickHouse/tests/integration/compose/docker_compose_minio.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': '6c39b51bdd1ee6641e47438265edd23f4a911da06c476fb341022c357974cf87'}, host_config={'NetworkMode': 'rootteststorages3queue_default', 'VolumesFrom': [], 'Binds': [], 'PortBindings': {}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}}, networking_config={'EndpointsConfig': {'rootteststorages3queue_default': {'Aliases': ['resolver'], 'IPAMConfig': {}}}}) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': '669ac9f6f6588e504e8df32560887202f2f63cd3d309131fa32fcc57bb52447b', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Warnings': []} (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('669ac9f6f6588e504e8df32560887202f2f63cd3d309131fa32fcc57bb52447b') (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': '888a29d3b59ff7a25b04e2960a4a87aa64a51a2840dae08bca06f77c604907b9', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Warnings': []} (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('888a29d3b59ff7a25b04e2960a4a87aa64a51a2840dae08bca06f77c604907b9') (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Args': ['server', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: '--console-address', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: '127.0.0.1:19001', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: '--address', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: ':9001', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: '--certs-dir', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: '/certs', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: '/data1-1'], (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('888a29d3b59ff7a25b04e2960a4a87aa64a51a2840dae08bca06f77c604907b9', 'rootteststorages3queue_default') (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Args': [], (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Cmd': ['python3'], (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'Env': ['PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr: 'GPG_KEY=7169605F62C751356D054A26A821E680E5FA6305', (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('669ac9f6f6588e504e8df32560887202f2f63cd3d309131fa32fcc57bb52447b', 'rootteststorages3queue_default') (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('669ac9f6f6588e504e8df32560887202f2f63cd3d309131fa32fcc57bb52447b', 'rootteststorages3queue_default', aliases=['669ac9f6f658', 'resolver'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('888a29d3b59ff7a25b04e2960a4a87aa64a51a2840dae08bca06f77c604907b9', 'rootteststorages3queue_default', aliases=['888a29d3b59f', 'minio1'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('669ac9f6f6588e504e8df32560887202f2f63cd3d309131fa32fcc57bb52447b') (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('888a29d3b59ff7a25b04e2960a4a87aa64a51a2840dae08bca06f77c604907b9') (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='rootteststorages3queue', service='resolver', number=1) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_resolver_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='rootteststorages3queue', service='minio1', number=1) (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_minio1_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:12 [ 514 ] INFO : Trying to connect to Minio... (cluster.py:3097, start) 2026-07-18 08:28:12 [ 514 ] DEBUG : get_instance_ip instance_name=minio1 (cluster.py:2137, get_instance_ip) 2026-07-18 08:28:12 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/rootteststorages3queue_minio1_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:28:12 [ 514 ] DEBUG : get_instance_ip instance_name=proxy1 (cluster.py:2137, get_instance_ip) 2026-07-18 08:28:12 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/rootteststorages3queue_proxy1_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:28:12 [ 514 ] DEBUG : Starting new HTTP connection (1): 172.16.1.8:9001 (connectionpool.py:245, _new_conn) 2026-07-18 08:28:12 [ 514 ] DEBUG : Incremented Retry for (url='/'): Retry(total=2, connect=None, read=None, redirect=None, status=None) (retry.py:517, increment) 2026-07-18 08:28:12 [ 514 ] WARNING : Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')': / (connectionpool.py:872, urlopen) 2026-07-18 08:28:12 [ 514 ] DEBUG : Starting new HTTP connection (2): 172.16.1.8:9001 (connectionpool.py:245, _new_conn) 2026-07-18 08:28:12 [ 514 ] DEBUG : Incremented Retry for (url='/'): Retry(total=1, connect=None, read=None, redirect=None, status=None) (retry.py:517, increment) 2026-07-18 08:28:12 [ 514 ] WARNING : Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')': / (connectionpool.py:872, urlopen) 2026-07-18 08:28:12 [ 514 ] DEBUG : Starting new HTTP connection (3): 172.16.1.8:9001 (connectionpool.py:245, _new_conn) 2026-07-18 08:28:12 [ 514 ] DEBUG : Incremented Retry for (url='/'): Retry(total=0, connect=None, read=None, redirect=None, status=None) (retry.py:517, increment) 2026-07-18 08:28:12 [ 514 ] WARNING : Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')': / (connectionpool.py:872, urlopen) 2026-07-18 08:28:12 [ 514 ] DEBUG : Starting new HTTP connection (4): 172.16.1.8:9001 (connectionpool.py:245, _new_conn) 2026-07-18 08:28:12 [ 514 ] DEBUG : Can't connect to Minio: HTTPConnectionPool(host='172.16.1.8', port=9001): Max retries exceeded with url: / (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) (cluster.py:2671, wait_minio_to_start) 2026-07-18 08:28:13 [ 514 ] DEBUG : Starting new HTTP connection (5): 172.16.1.8:9001 (connectionpool.py:245, _new_conn) 2026-07-18 08:28:13 [ 514 ] DEBUG : http://172.16.1.8:9001 "GET / HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:13 [ 514 ] DEBUG : Connected to Minio. (cluster.py:2651, wait_minio_to_start) 2026-07-18 08:28:13 [ 514 ] DEBUG : http://172.16.1.8:9001 "GET /root?location= HTTP/1.1" 404 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:13 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:13 [ 514 ] DEBUG : S3 bucket 'root' created (cluster.py:2666, wait_minio_to_start) 2026-07-18 08:28:13 [ 514 ] DEBUG : http://172.16.1.8:9001 "GET /root2?location= HTTP/1.1" 404 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:13 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root2 HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:13 [ 514 ] DEBUG : S3 bucket 'root2' created (cluster.py:2666, wait_minio_to_start) 2026-07-18 08:28:13 [ 514 ] INFO : Trying to create Azurite instance by command docker-compose --env-file /ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/.env --project-name rootteststorages3queue --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml --verbose up -d (cluster.py:3102, start) 2026-07-18 08:28:13 [ 514 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/.env', '--project-name', 'rootteststorages3queue', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml', '--verbose', 'up', '-d'] (cluster.py:113, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.config.config.find: Using configuration files: /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.docker_client.get_client: docker-compose version 1.29.2, build unknown (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:docker-py version: (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:CPython version: 3.10.12 (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:OpenSSL version: OpenSSL 3.0.2 15 Mar 2022 (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.docker_client.get_client: Docker base_url: http+docker://localhost (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.docker_client.get_client: Docker version: Platform={'Name': 'Docker Engine - Community'}, Components=[{'Name': 'Engine', 'Version': '23.0.6', 'Details': {'ApiVersion': '1.42', 'Arch': 'amd64', 'BuildTime': '2023-05-05T21:18:13.000000000+00:00', 'Experimental': 'false', 'GitCommit': '9dbdbd4', 'GoVersion': 'go1.19.9', 'KernelVersion': '5.15.0-185-generic', 'MinAPIVersion': '1.12', 'Os': 'linux'}}, {'Name': 'containerd', 'Version': 'v2.2.6', 'Details': {'GitCommit': '11ce9d5f3c68c941867e82890e93e815c1304f1b'}}, {'Name': 'runc', 'Version': '1.3.6', 'Details': {'GitCommit': 'v1.3.6-0-g491b69ba'}}, {'Name': 'docker-init', 'Version': '0.19.0', 'Details': {'GitCommit': 'de40ad0'}}], Version=23.0.6, ApiVersion=1.42, MinAPIVersion=1.12, GitCommit=9dbdbd4, GoVersion=go1.19.9, Os=linux, Arch=amd64, KernelVersion=5.15.0-185-generic, BuildTime=2023-05-05T21:18:13.000000000+00:00 (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('rootteststorages3queue_default') (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network -> {'Attachable': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'ConfigFrom': {'Network': ''}, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'ConfigOnly': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Containers': {'41142ac2c4f637cf206c408e54b5d85940af520ec363a99cae85c7b00d0bb765': {'EndpointID': '9b7a2f79cb2d82ab1ada98a8211486e8193dd4c3726a12e3886c78e71a9004ef', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'IPv4Address': '172.16.1.6/24', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'IPv6Address': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'MacAddress': '02:42:ac:10:01:06', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Name': 'rootteststorages3queue_proxy1_1'}, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: '5852136cd9392dd79f7ee8d6320819eaf5cd5e2b033fb675c937845cb486c533': {'EndpointID': '474d421d202ff1eeaf9fa1c48e758ea19e9cc8b915b331742c7deaee36351542', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'IPv4Address': '172.16.1.3/24', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_volume <- ('rootteststorages3queue_data1-1') (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_volume -> {'CreatedAt': '2026-07-18T08:28:11Z', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Driver': 'local', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Labels': {'com.docker.compose.project': 'rootteststorages3queue', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'com.docker.compose.version': '1.29.2', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'com.docker.compose.volume': 'data1-1'}, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Mountpoint': '/var/lib/docker/volumes/rootteststorages3queue_data1-1/_data', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Name': 'rootteststorages3queue_data1-1', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Options': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Scope': 'local'} (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker info <- () (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker info -> {'Architecture': 'x86_64', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'BridgeNfIp6tables': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'BridgeNfIptables': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'CPUSet': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'CPUShares': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'CgroupDriver': 'cgroupfs', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'CgroupVersion': '2', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'ContainerdCommit': {'Expected': '11ce9d5f3c68c941867e82890e93e815c1304f1b', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'ID': '11ce9d5f3c68c941867e82890e93e815c1304f1b'}, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Containers': 7, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('rootteststorages3queue_default') (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network -> {'Attachable': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'ConfigFrom': {'Network': ''}, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'ConfigOnly': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Containers': {'41142ac2c4f637cf206c408e54b5d85940af520ec363a99cae85c7b00d0bb765': {'EndpointID': '9b7a2f79cb2d82ab1ada98a8211486e8193dd4c3726a12e3886c78e71a9004ef', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'IPv4Address': '172.16.1.6/24', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'IPv6Address': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'MacAddress': '02:42:ac:10:01:06', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Name': 'rootteststorages3queue_proxy1_1'}, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: '5852136cd9392dd79f7ee8d6320819eaf5cd5e2b033fb675c937845cb486c533': {'EndpointID': '474d421d202ff1eeaf9fa1c48e758ea19e9cc8b915b331742c7deaee36351542', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'IPv4Address': '172.16.1.3/24', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_volume <- ('rootteststorages3queue_data1-1') (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_volume -> {'CreatedAt': '2026-07-18T08:28:11Z', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Driver': 'local', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Labels': {'com.docker.compose.project': 'rootteststorages3queue', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'com.docker.compose.version': '1.29.2', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'com.docker.compose.volume': 'data1-1'}, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Mountpoint': '/var/lib/docker/volumes/rootteststorages3queue_data1-1/_data', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Name': 'rootteststorages3queue_data1-1', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Options': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Scope': 'local'} (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_volume <- ('rootteststorages3queue_data1-1') (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_volume -> {'CreatedAt': '2026-07-18T08:28:11Z', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Driver': 'local', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Labels': {'com.docker.compose.project': 'rootteststorages3queue', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'com.docker.compose.version': '1.29.2', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'com.docker.compose.volume': 'data1-1'}, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Mountpoint': '/var/lib/docker/volumes/rootteststorages3queue_data1-1/_data', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Name': 'rootteststorages3queue_data1-1', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Options': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Scope': 'local'} (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 7 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 7 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('5852136cd9392dd79f7ee8d6320819eaf5cd5e2b033fb675c937845cb486c533') (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: '--config=/etc/clickhouse-keeper/keeper_config1.xml', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Cmd': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('888a29d3b59ff7a25b04e2960a4a87aa64a51a2840dae08bca06f77c604907b9') (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Args': ['server', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: '--console-address', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: '127.0.0.1:19001', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: '--address', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: ':9001', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: '--certs-dir', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: '/certs', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: '/data1-1'], (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('41142ac2c4f637cf206c408e54b5d85940af520ec363a99cae85c7b00d0bb765') (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Args': ['/run.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Cmd': ['/run.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Entrypoint': ['/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Env': ['PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'NGINX_VERSION=1.27.5', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('d14c2cfabc1d0ebd0a98f90504d13e0761310b227a4620de0c83df05cf85a99d') (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Args': ['/run.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Cmd': ['/run.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Entrypoint': ['/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Env': ['PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'NGINX_VERSION=1.27.5', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('732ce5e607697aa126a740328b276089d0d2ce5a04462823f13ffa579bfc83ec') (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: '--config=/etc/clickhouse-keeper/keeper_config2.xml', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Cmd': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('629e59ca2bd58378e382b49b054100ea9106be98ddd11a6d2bd1ffbe1fcde657') (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: '--config=/etc/clickhouse-keeper/keeper_config3.xml', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Cmd': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('669ac9f6f6588e504e8df32560887202f2f63cd3d309131fa32fcc57bb52447b') (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Args': [], (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Cmd': ['python3'], (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Env': ['PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'GPG_KEY=7169605F62C751356D054A26A821E680E5FA6305', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.project.find_orphan_containers: Found orphan containers (rootteststorages3queue_zoo1_1, rootteststorages3queue_minio1_1, rootteststorages3queue_proxy1_1, rootteststorages3queue_proxy2_1, rootteststorages3queue_zoo2_1, rootteststorages3queue_zoo3_1, rootteststorages3queue_resolver_1) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up. (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=azurite1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=azurite1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/azurite:3.35.0') (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Cmd': ['azurite', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: '-l', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: '/data', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=azurite1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=azurite1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {} (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=azurite1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=azurite1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_azurite1_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='rootteststorages3queue', service='azurite1', number=1)} (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='rootteststorages3queue', service='azurite1', number=1) (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/azurite:3.35.0') (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Cmd': ['azurite', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: '-l', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: '/data', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/azurite:3.35.0') (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Cmd': ['azurite', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: '-l', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: '/data', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: 2ece5e07b89b0f4e7b5e8076ad1e1b0717274882ab654de56a4c7613ded50eb6 (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={'30000/tcp': ['30000']}, binds=['rootteststorages3queue_data1-1:/data1:rw'], volumes_from=[], privileged=False, network_mode='rootteststorages3queue_default', devices=None, device_requests=None, dns=None, dns_opt=None, dns_search=None, restart_policy=None, runtime=None, cap_add=None, cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=None, ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=None, device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': ['rootteststorages3queue_data1-1:/data1:rw'], (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Links': [], (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'NetworkMode': 'rootteststorages3queue_default', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'PortBindings': {'30000/tcp': [{'HostIp': '', 'HostPort': '30000'}]}, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'VolumesFrom': []} (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (command='azurite-blob --blobHost 0.0.0.0 --blobPort 30000 --debug /azurite_log', image='altinityinfra/azurite:3.35.0', ports=[('30000', 'tcp')], volumes={'/data1': {}}, name='rootteststorages3queue_azurite1_1', detach=True, environment=[], labels={'com.docker.compose.project': 'rootteststorages3queue', 'com.docker.compose.service': 'azurite1', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/ClickHouse/tests/integration/compose', 'com.docker.compose.project.config_files': '/ClickHouse/tests/integration/compose/docker_compose_azurite.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': '2ece5e07b89b0f4e7b5e8076ad1e1b0717274882ab654de56a4c7613ded50eb6'}, host_config={'NetworkMode': 'rootteststorages3queue_default', 'VolumesFrom': [], 'Binds': ['rootteststorages3queue_data1-1:/data1:rw'], 'PortBindings': {'30000/tcp': [{'HostIp': '', 'HostPort': '30000'}]}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}}, networking_config={'EndpointsConfig': {'rootteststorages3queue_default': {'Aliases': ['azurite1'], 'IPAMConfig': {}}}}) (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': 'ceb4658d869eed946ee2584fb25a085a860667930370d6ab311bec9d05d7de41', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Warnings': []} (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('ceb4658d869eed946ee2584fb25a085a860667930370d6ab311bec9d05d7de41') (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Args': ['azurite-blob', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: '--blobHost', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: '0.0.0.0', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: '--blobPort', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: '30000', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: '--debug', (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: '/azurite_log'], (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('ceb4658d869eed946ee2584fb25a085a860667930370d6ab311bec9d05d7de41', 'rootteststorages3queue_default') (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('ceb4658d869eed946ee2584fb25a085a860667930370d6ab311bec9d05d7de41', 'rootteststorages3queue_default', aliases=['azurite1', 'ceb4658d869e'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('ceb4658d869eed946ee2584fb25a085a860667930370d6ab311bec9d05d7de41') (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='rootteststorages3queue', service='azurite1', number=1) (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_azurite1_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:13 [ 514 ] INFO : Trying to connect to Azurite (cluster.py:3118, start) 2026-07-18 08:28:14 [ 514 ] INFO : Request URL: 'http://127.0.0.1:30000/devstoreaccount1/?restype=REDACTED&comp=REDACTED' Request method: 'GET' Request headers: 'x-ms-version': 'REDACTED' 'Accept': 'application/xml' 'User-Agent': 'azsdk-python-storage-blob/12.19.1 Python/3.10.12 (Linux-5.15.0-185-generic-x86_64-with-glibc2.35)' 'x-ms-date': 'REDACTED' 'x-ms-client-request-id': '9eecb440-8282-11f1-90de-9a498876852b' 'Authorization': 'REDACTED' No body was attached to the request (_universal.py:513, on_request) 2026-07-18 08:28:14 [ 514 ] DEBUG : Starting new HTTP connection (1): 127.0.0.1:30000 (connectionpool.py:245, _new_conn) 2026-07-18 08:28:14 [ 514 ] DEBUG : http://127.0.0.1:30000 "GET /devstoreaccount1/?restype=account&comp=properties HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:14 [ 514 ] INFO : Response status: 200 Response headers: 'Server': 'Azurite-Blob/3.35.0' 'x-ms-client-request-id': '9eecb440-8282-11f1-90de-9a498876852b' 'x-ms-request-id': 'f8dc88b6-872b-4957-b571-96acfbcc9532' 'x-ms-version': 'REDACTED' 'date': 'Sat, 18 Jul 2026 08:28:14 GMT' 'x-ms-sku-name': 'REDACTED' 'x-ms-account-kind': 'REDACTED' 'x-ms-is-hns-enabled': 'REDACTED' 'Connection': 'keep-alive' 'Keep-Alive': 'REDACTED' 'Content-Length': '0' (_universal.py:549, on_response) 2026-07-18 08:28:14 [ 514 ] DEBUG : {'client_request_id': '9eecb440-8282-11f1-90de-9a498876852b', 'request_id': 'f8dc88b6-872b-4957-b571-96acfbcc9532', 'version': '2025-11-05', 'date': datetime.datetime(2026, 7, 18, 8, 28, 14, tzinfo=datetime.timezone.utc), 'sku_name': 'Standard_RAGRS', 'account_kind': 'StorageV2', 'is_hns_enabled': False} (cluster.py:2699, wait_azurite_to_start) 2026-07-18 08:28:14 [ 514 ] DEBUG : ('Trying to create ClickHouse instance by command %s', 'docker-compose --env-file /ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/.env --project-name rootteststorages3queue --file /ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml --file /ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance2/docker-compose.yml --file /ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/old_instance/docker-compose.yml --file /ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance_too_many_parts/docker-compose.yml --file /ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance_24.5/docker-compose.yml up -d --no-recreate') (cluster.py:3150, start) 2026-07-18 08:28:14 [ 514 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/.env', '--project-name', 'rootteststorages3queue', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance2/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/old_instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance_too_many_parts/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance_24.5/docker-compose.yml', 'up', '-d', '--no-recreate'] (cluster.py:113, run_and_check) 2026-07-18 08:28:16 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_instance_too_many_parts_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:16 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_instance_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:16 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_instance_24.5_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:16 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_old_instance_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:16 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_instance2_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:16 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_instance_24.5_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:16 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_instance_too_many_parts_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:16 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_instance_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:16 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_old_instance_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:16 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_instance2_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:16 [ 514 ] DEBUG : ClickHouse instance created (cluster.py:3158, start) 2026-07-18 08:28:16 [ 514 ] DEBUG : get_instance_ip instance_name=instance (cluster.py:2137, get_instance_ip) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/rootteststorages3queue_instance_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : Waiting for ClickHouse start in instance, ip: 172.16.1.11... (cluster.py:3165, start) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/rootteststorages3queue_instance_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/7e8207cae8d2eded12c1014ea91bc578053d72af27bd9679f2e265b343ecd718/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/7e8207cae8d2eded12c1014ea91bc578053d72af27bd9679f2e265b343ecd718/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/7e8207cae8d2eded12c1014ea91bc578053d72af27bd9679f2e265b343ecd718/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : ClickHouse instance started (cluster.py:3169, start) 2026-07-18 08:28:16 [ 514 ] DEBUG : get_instance_ip instance_name=instance2 (cluster.py:2137, get_instance_ip) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/rootteststorages3queue_instance2_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : Waiting for ClickHouse start in instance2, ip: 172.16.1.14... (cluster.py:3165, start) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/rootteststorages3queue_instance2_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/633341f02470c05867ce1a116fda958e2ac0de79575c21761701b271c304d89d/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : ClickHouse instance2 started (cluster.py:3169, start) 2026-07-18 08:28:16 [ 514 ] DEBUG : get_instance_ip instance_name=old_instance (cluster.py:2137, get_instance_ip) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/rootteststorages3queue_old_instance_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : Waiting for ClickHouse start in old_instance, ip: 172.16.1.13... (cluster.py:3165, start) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/rootteststorages3queue_old_instance_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/5dcb7f43e90139739a5768b115d408c6a1c5f565cfd7be1dfc403c8869715e75/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : ClickHouse old_instance started (cluster.py:3169, start) 2026-07-18 08:28:16 [ 514 ] DEBUG : get_instance_ip instance_name=instance_too_many_parts (cluster.py:2137, get_instance_ip) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/rootteststorages3queue_instance_too_many_parts_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : Waiting for ClickHouse start in instance_too_many_parts, ip: 172.16.1.12... (cluster.py:3165, start) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/rootteststorages3queue_instance_too_many_parts_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/0c3d5ae3bce5d8f8874956e191bb33304e0a631d68c3a7e061230cb168c4e167/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : ClickHouse instance_too_many_parts started (cluster.py:3169, start) 2026-07-18 08:28:16 [ 514 ] DEBUG : get_instance_ip instance_name=instance_24.5 (cluster.py:2137, get_instance_ip) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/rootteststorages3queue_instance_24.5_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : Waiting for ClickHouse start in instance_24.5, ip: 172.16.1.10... (cluster.py:3165, start) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/rootteststorages3queue_instance_24.5_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/e4f37c5a1bb949c8356cff9a8b27ac8946dbe0755c6da0160846f8d8e690a975/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : ClickHouse instance_24.5 started (cluster.py:3169, start) 2026-07-18 08:28:16 [ 514 ] INFO : Cluster started (test.py:145, started_cluster) 2026-07-18 08:28:16 [ 514 ] INFO : Request URL: 'http://127.0.0.1:30000/devstoreaccount1/cont?restype=REDACTED' Request method: 'PUT' Request headers: 'x-ms-version': 'REDACTED' 'Accept': 'application/xml' 'User-Agent': 'azsdk-python-storage-blob/12.19.1 Python/3.10.12 (Linux-5.15.0-185-generic-x86_64-with-glibc2.35)' 'x-ms-date': 'REDACTED' 'x-ms-client-request-id': '9fe1aae0-8282-11f1-90de-9a498876852b' 'Authorization': 'REDACTED' No body was attached to the request (_universal.py:513, on_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://127.0.0.1:30000 "PUT /devstoreaccount1/cont?restype=container HTTP/1.1" 201 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] INFO : Response status: 201 Response headers: 'Server': 'Azurite-Blob/3.35.0' 'etag': '"0x2545DC6625A9E60"' 'last-modified': 'Sat, 18 Jul 2026 08:28:16 GMT' 'x-ms-client-request-id': '9fe1aae0-8282-11f1-90de-9a498876852b' 'x-ms-request-id': 'e2b08935-8ddf-4ab2-95fe-1c8ed490c181' 'x-ms-version': 'REDACTED' 'Date': 'Sat, 18 Jul 2026 08:28:16 GMT' 'Connection': 'keep-alive' 'Keep-Alive': 'REDACTED' 'Content-Length': '0' (_universal.py:549, on_response) 2026-07-18 08:28:16 [ 514 ] DEBUG : Executing query DROP DATABASE IF EXISTS test; CREATE DATABASE test; on instance (cluster.py:3606, query) 2026-07-18 08:28:16 [ 514 ] DEBUG : Executing query DROP DATABASE IF EXISTS test; CREATE DATABASE test; on instance2 (cluster.py:3606, query) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "GET /root?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix= HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) ----------------------------- Captured stdout call ----------------------------- Generating files: [('test_shards_distributed_ordered_8_data/test_0.csv', 0), ('test_shards_distributed_ordered_8_data/test_1.csv', 1), ('test_shards_distributed_ordered_8_data/test_10.csv', 10), ('test_shards_distributed_ordered_8_data/test_100.csv', 100), ('test_shards_distributed_ordered_8_data/test_101.csv', 101), ('test_shards_distributed_ordered_8_data/test_102.csv', 102), ('test_shards_distributed_ordered_8_data/test_103.csv', 103), ('test_shards_distributed_ordered_8_data/test_104.csv', 104), ('test_shards_distributed_ordered_8_data/test_105.csv', 105), ('test_shards_distributed_ordered_8_data/test_106.csv', 106), ('test_shards_distributed_ordered_8_data/test_107.csv', 107), ('test_shards_distributed_ordered_8_data/test_108.csv', 108), ('test_shards_distributed_ordered_8_data/test_109.csv', 109), ('test_shards_distributed_ordered_8_data/test_11.csv', 11), ('test_shards_distributed_ordered_8_data/test_110.csv', 110), ('test_shards_distributed_ordered_8_data/test_111.csv', 111), ('test_shards_distributed_ordered_8_data/test_112.csv', 112), ('test_shards_distributed_ordered_8_data/test_113.csv', 113), ('test_shards_distributed_ordered_8_data/test_114.csv', 114), ('test_shards_distributed_ordered_8_data/test_115.csv', 115), ('test_shards_distributed_ordered_8_data/test_116.csv', 116), ('test_shards_distributed_ordered_8_data/test_117.csv', 117), ('test_shards_distributed_ordered_8_data/test_118.csv', 118), ('test_shards_distributed_ordered_8_data/test_119.csv', 119), ('test_shards_distributed_ordered_8_data/test_12.csv', 12), ('test_shards_distributed_ordered_8_data/test_120.csv', 120), ('test_shards_distributed_ordered_8_data/test_121.csv', 121), ('test_shards_distributed_ordered_8_data/test_122.csv', 122), ('test_shards_distributed_ordered_8_data/test_123.csv', 123), ('test_shards_distributed_ordered_8_data/test_124.csv', 124), ('test_shards_distributed_ordered_8_data/test_125.csv', 125), ('test_shards_distributed_ordered_8_data/test_126.csv', 126), ('test_shards_distributed_ordered_8_data/test_127.csv', 127), ('test_shards_distributed_ordered_8_data/test_128.csv', 128), ('test_shards_distributed_ordered_8_data/test_129.csv', 129), ('test_shards_distributed_ordered_8_data/test_13.csv', 13), ('test_shards_distributed_ordered_8_data/test_130.csv', 130), ('test_shards_distributed_ordered_8_data/test_131.csv', 131), ('test_shards_distributed_ordered_8_data/test_132.csv', 132), ('test_shards_distributed_ordered_8_data/test_133.csv', 133), ('test_shards_distributed_ordered_8_data/test_134.csv', 134), ('test_shards_distributed_ordered_8_data/test_135.csv', 135), ('test_shards_distributed_ordered_8_data/test_136.csv', 136), ('test_shards_distributed_ordered_8_data/test_137.csv', 137), ('test_shards_distributed_ordered_8_data/test_138.csv', 138), ('test_shards_distributed_ordered_8_data/test_139.csv', 139), ('test_shards_distributed_ordered_8_data/test_14.csv', 14), ('test_shards_distributed_ordered_8_data/test_140.csv', 140), ('test_shards_distributed_ordered_8_data/test_141.csv', 141), ('test_shards_distributed_ordered_8_data/test_142.csv', 142), ('test_shards_distributed_ordered_8_data/test_143.csv', 143), ('test_shards_distributed_ordered_8_data/test_144.csv', 144), ('test_shards_distributed_ordered_8_data/test_145.csv', 145), ('test_shards_distributed_ordered_8_data/test_146.csv', 146), ('test_shards_distributed_ordered_8_data/test_147.csv', 147), ('test_shards_distributed_ordered_8_data/test_148.csv', 148), ('test_shards_distributed_ordered_8_data/test_149.csv', 149), ('test_shards_distributed_ordered_8_data/test_15.csv', 15), ('test_shards_distributed_ordered_8_data/test_150.csv', 150), ('test_shards_distributed_ordered_8_data/test_151.csv', 151), ('test_shards_distributed_ordered_8_data/test_152.csv', 152), ('test_shards_distributed_ordered_8_data/test_153.csv', 153), ('test_shards_distributed_ordered_8_data/test_154.csv', 154), ('test_shards_distributed_ordered_8_data/test_155.csv', 155), ('test_shards_distributed_ordered_8_data/test_156.csv', 156), ('test_shards_distributed_ordered_8_data/test_157.csv', 157), ('test_shards_distributed_ordered_8_data/test_158.csv', 158), ('test_shards_distributed_ordered_8_data/test_159.csv', 159), ('test_shards_distributed_ordered_8_data/test_16.csv', 16), ('test_shards_distributed_ordered_8_data/test_160.csv', 160), ('test_shards_distributed_ordered_8_data/test_161.csv', 161), ('test_shards_distributed_ordered_8_data/test_162.csv', 162), ('test_shards_distributed_ordered_8_data/test_163.csv', 163), ('test_shards_distributed_ordered_8_data/test_164.csv', 164), ('test_shards_distributed_ordered_8_data/test_165.csv', 165), ('test_shards_distributed_ordered_8_data/test_166.csv', 166), ('test_shards_distributed_ordered_8_data/test_167.csv', 167), ('test_shards_distributed_ordered_8_data/test_168.csv', 168), ('test_shards_distributed_ordered_8_data/test_169.csv', 169), ('test_shards_distributed_ordered_8_data/test_17.csv', 17), ('test_shards_distributed_ordered_8_data/test_170.csv', 170), ('test_shards_distributed_ordered_8_data/test_171.csv', 171), ('test_shards_distributed_ordered_8_data/test_172.csv', 172), ('test_shards_distributed_ordered_8_data/test_173.csv', 173), ('test_shards_distributed_ordered_8_data/test_174.csv', 174), ('test_shards_distributed_ordered_8_data/test_175.csv', 175), ('test_shards_distributed_ordered_8_data/test_176.csv', 176), ('test_shards_distributed_ordered_8_data/test_177.csv', 177), ('test_shards_distributed_ordered_8_data/test_178.csv', 178), ('test_shards_distributed_ordered_8_data/test_179.csv', 179), ('test_shards_distributed_ordered_8_data/test_18.csv', 18), ('test_shards_distributed_ordered_8_data/test_180.csv', 180), ('test_shards_distributed_ordered_8_data/test_181.csv', 181), ('test_shards_distributed_ordered_8_data/test_182.csv', 182), ('test_shards_distributed_ordered_8_data/test_183.csv', 183), ('test_shards_distributed_ordered_8_data/test_184.csv', 184), ('test_shards_distributed_ordered_8_data/test_185.csv', 185), ('test_shards_distributed_ordered_8_data/test_186.csv', 186), ('test_shards_distributed_ordered_8_data/test_187.csv', 187), ('test_shards_distributed_ordered_8_data/test_188.csv', 188), ('test_shards_distributed_ordered_8_data/test_189.csv', 189), ('test_shards_distributed_ordered_8_data/test_19.csv', 19), ('test_shards_distributed_ordered_8_data/test_190.csv', 190), ('test_shards_distributed_ordered_8_data/test_191.csv', 191), ('test_shards_distributed_ordered_8_data/test_192.csv', 192), ('test_shards_distributed_ordered_8_data/test_193.csv', 193), ('test_shards_distributed_ordered_8_data/test_194.csv', 194), ('test_shards_distributed_ordered_8_data/test_195.csv', 195), ('test_shards_distributed_ordered_8_data/test_196.csv', 196), ('test_shards_distributed_ordered_8_data/test_197.csv', 197), ('test_shards_distributed_ordered_8_data/test_198.csv', 198), ('test_shards_distributed_ordered_8_data/test_199.csv', 199), ('test_shards_distributed_ordered_8_data/test_2.csv', 2), ('test_shards_distributed_ordered_8_data/test_20.csv', 20), ('test_shards_distributed_ordered_8_data/test_200.csv', 200), ('test_shards_distributed_ordered_8_data/test_201.csv', 201), ('test_shards_distributed_ordered_8_data/test_202.csv', 202), ('test_shards_distributed_ordered_8_data/test_203.csv', 203), ('test_shards_distributed_ordered_8_data/test_204.csv', 204), ('test_shards_distributed_ordered_8_data/test_205.csv', 205), ('test_shards_distributed_ordered_8_data/test_206.csv', 206), ('test_shards_distributed_ordered_8_data/test_207.csv', 207), ('test_shards_distributed_ordered_8_data/test_208.csv', 208), ('test_shards_distributed_ordered_8_data/test_209.csv', 209), ('test_shards_distributed_ordered_8_data/test_21.csv', 21), ('test_shards_distributed_ordered_8_data/test_210.csv', 210), ('test_shards_distributed_ordered_8_data/test_211.csv', 211), ('test_shards_distributed_ordered_8_data/test_212.csv', 212), ('test_shards_distributed_ordered_8_data/test_213.csv', 213), ('test_shards_distributed_ordered_8_data/test_214.csv', 214), ('test_shards_distributed_ordered_8_data/test_215.csv', 215), ('test_shards_distributed_ordered_8_data/test_216.csv', 216), ('test_shards_distributed_ordered_8_data/test_217.csv', 217), ('test_shards_distributed_ordered_8_data/test_218.csv', 218), ('test_shards_distributed_ordered_8_data/test_219.csv', 219), ('test_shards_distributed_ordered_8_data/test_22.csv', 22), ('test_shards_distributed_ordered_8_data/test_220.csv', 220), ('test_shards_distributed_ordered_8_data/test_221.csv', 221), ('test_shards_distributed_ordered_8_data/test_222.csv', 222), ('test_shards_distributed_ordered_8_data/test_223.csv', 223), ('test_shards_distributed_ordered_8_data/test_224.csv', 224), ('test_shards_distributed_ordered_8_data/test_225.csv', 225), ('test_shards_distributed_ordered_8_data/test_226.csv', 226), ('test_shards_distributed_ordered_8_data/test_227.csv', 227), ('test_shards_distributed_ordered_8_data/test_228.csv', 228), ('test_shards_distributed_ordered_8_data/test_229.csv', 229), ('test_shards_distributed_ordered_8_data/test_23.csv', 23), ('test_shards_distributed_ordered_8_data/test_230.csv', 230), ('test_shards_distributed_ordered_8_data/test_231.csv', 231), ('test_shards_distributed_ordered_8_data/test_232.csv', 232), ('test_shards_distributed_ordered_8_data/test_233.csv', 233), ('test_shards_distributed_ordered_8_data/test_234.csv', 234), ('test_shards_distributed_ordered_8_data/test_235.csv', 235), ('test_shards_distributed_ordered_8_data/test_236.csv', 236), ('test_shards_distributed_ordered_8_data/test_237.csv', 237), ('test_shards_distributed_ordered_8_data/test_238.csv', 238), ('test_shards_distributed_ordered_8_data/test_239.csv', 239), ('test_shards_distributed_ordered_8_data/test_24.csv', 24), ('test_shards_distributed_ordered_8_data/test_240.csv', 240), ('test_shards_distributed_ordered_8_data/test_241.csv', 241), ('test_shards_distributed_ordered_8_data/test_242.csv', 242), ('test_shards_distributed_ordered_8_data/test_243.csv', 243), ('test_shards_distributed_ordered_8_data/test_244.csv', 244), ('test_shards_distributed_ordered_8_data/test_245.csv', 245), ('test_shards_distributed_ordered_8_data/test_246.csv', 246), ('test_shards_distributed_ordered_8_data/test_247.csv', 247), ('test_shards_distributed_ordered_8_data/test_248.csv', 248), ('test_shards_distributed_ordered_8_data/test_249.csv', 249), ('test_shards_distributed_ordered_8_data/test_25.csv', 25), ('test_shards_distributed_ordered_8_data/test_250.csv', 250), ('test_shards_distributed_ordered_8_data/test_251.csv', 251), ('test_shards_distributed_ordered_8_data/test_252.csv', 252), ('test_shards_distributed_ordered_8_data/test_253.csv', 253), ('test_shards_distributed_ordered_8_data/test_254.csv', 254), ('test_shards_distributed_ordered_8_data/test_255.csv', 255), ('test_shards_distributed_ordered_8_data/test_256.csv', 256), ('test_shards_distributed_ordered_8_data/test_257.csv', 257), ('test_shards_distributed_ordered_8_data/test_258.csv', 258), ('test_shards_distributed_ordered_8_data/test_259.csv', 259), ('test_shards_distributed_ordered_8_data/test_26.csv', 26), ('test_shards_distributed_ordered_8_data/test_260.csv', 260), ('test_shards_distributed_ordered_8_data/test_261.csv', 261), ('test_shards_distributed_ordered_8_data/test_262.csv', 262), ('test_shards_distributed_ordered_8_data/test_263.csv', 263), ('test_shards_distributed_ordered_8_data/test_264.csv', 264), ('test_shards_distributed_ordered_8_data/test_265.csv', 265), ('test_shards_distributed_ordered_8_data/test_266.csv', 266), ('test_shards_distributed_ordered_8_data/test_267.csv', 267), ('test_shards_distributed_ordered_8_data/test_268.csv', 268), ('test_shards_distributed_ordered_8_data/test_269.csv', 269), ('test_shards_distributed_ordered_8_data/test_27.csv', 27), ('test_shards_distributed_ordered_8_data/test_270.csv', 270), ('test_shards_distributed_ordered_8_data/test_271.csv', 271), ('test_shards_distributed_ordered_8_data/test_272.csv', 272), ('test_shards_distributed_ordered_8_data/test_273.csv', 273), ('test_shards_distributed_ordered_8_data/test_274.csv', 274), ('test_shards_distributed_ordered_8_data/test_275.csv', 275), ('test_shards_distributed_ordered_8_data/test_276.csv', 276), ('test_shards_distributed_ordered_8_data/test_277.csv', 277), ('test_shards_distributed_ordered_8_data/test_278.csv', 278), ('test_shards_distributed_ordered_8_data/test_279.csv', 279), ('test_shards_distributed_ordered_8_data/test_28.csv', 28), ('test_shards_distributed_ordered_8_data/test_280.csv', 280), ('test_shards_distributed_ordered_8_data/test_281.csv', 281), ('test_shards_distributed_ordered_8_data/test_282.csv', 282), ('test_shards_distributed_ordered_8_data/test_283.csv', 283), ('test_shards_distributed_ordered_8_data/test_284.csv', 284), ('test_shards_distributed_ordered_8_data/test_285.csv', 285), ('test_shards_distributed_ordered_8_data/test_286.csv', 286), ('test_shards_distributed_ordered_8_data/test_287.csv', 287), ('test_shards_distributed_ordered_8_data/test_288.csv', 288), ('test_shards_distributed_ordered_8_data/test_289.csv', 289), ('test_shards_distributed_ordered_8_data/test_29.csv', 29), ('test_shards_distributed_ordered_8_data/test_290.csv', 290), ('test_shards_distributed_ordered_8_data/test_291.csv', 291), ('test_shards_distributed_ordered_8_data/test_292.csv', 292), ('test_shards_distributed_ordered_8_data/test_293.csv', 293), ('test_shards_distributed_ordered_8_data/test_294.csv', 294), ('test_shards_distributed_ordered_8_data/test_295.csv', 295), ('test_shards_distributed_ordered_8_data/test_296.csv', 296), ('test_shards_distributed_ordered_8_data/test_297.csv', 297), ('test_shards_distributed_ordered_8_data/test_298.csv', 298), ('test_shards_distributed_ordered_8_data/test_299.csv', 299), ('test_shards_distributed_ordered_8_data/test_3.csv', 3), ('test_shards_distributed_ordered_8_data/test_30.csv', 30), ('test_shards_distributed_ordered_8_data/test_31.csv', 31), ('test_shards_distributed_ordered_8_data/test_32.csv', 32), ('test_shards_distributed_ordered_8_data/test_33.csv', 33), ('test_shards_distributed_ordered_8_data/test_34.csv', 34), ('test_shards_distributed_ordered_8_data/test_35.csv', 35), ('test_shards_distributed_ordered_8_data/test_36.csv', 36), ('test_shards_distributed_ordered_8_data/test_37.csv', 37), ('test_shards_distributed_ordered_8_data/test_38.csv', 38), ('test_shards_distributed_ordered_8_data/test_39.csv', 39), ('test_shards_distributed_ordered_8_data/test_4.csv', 4), ('test_shards_distributed_ordered_8_data/test_40.csv', 40), ('test_shards_distributed_ordered_8_data/test_41.csv', 41), ('test_shards_distributed_ordered_8_data/test_42.csv', 42), ('test_shards_distributed_ordered_8_data/test_43.csv', 43), ('test_shards_distributed_ordered_8_data/test_44.csv', 44), ('test_shards_distributed_ordered_8_data/test_45.csv', 45), ('test_shards_distributed_ordered_8_data/test_46.csv', 46), ('test_shards_distributed_ordered_8_data/test_47.csv', 47), ('test_shards_distributed_ordered_8_data/test_48.csv', 48), ('test_shards_distributed_ordered_8_data/test_49.csv', 49), ('test_shards_distributed_ordered_8_data/test_5.csv', 5), ('test_shards_distributed_ordered_8_data/test_50.csv', 50), ('test_shards_distributed_ordered_8_data/test_51.csv', 51), ('test_shards_distributed_ordered_8_data/test_52.csv', 52), ('test_shards_distributed_ordered_8_data/test_53.csv', 53), ('test_shards_distributed_ordered_8_data/test_54.csv', 54), ('test_shards_distributed_ordered_8_data/test_55.csv', 55), ('test_shards_distributed_ordered_8_data/test_56.csv', 56), ('test_shards_distributed_ordered_8_data/test_57.csv', 57), ('test_shards_distributed_ordered_8_data/test_58.csv', 58), ('test_shards_distributed_ordered_8_data/test_59.csv', 59), ('test_shards_distributed_ordered_8_data/test_6.csv', 6), ('test_shards_distributed_ordered_8_data/test_60.csv', 60), ('test_shards_distributed_ordered_8_data/test_61.csv', 61), ('test_shards_distributed_ordered_8_data/test_62.csv', 62), ('test_shards_distributed_ordered_8_data/test_63.csv', 63), ('test_shards_distributed_ordered_8_data/test_64.csv', 64), ('test_shards_distributed_ordered_8_data/test_65.csv', 65), ('test_shards_distributed_ordered_8_data/test_66.csv', 66), ('test_shards_distributed_ordered_8_data/test_67.csv', 67), ('test_shards_distributed_ordered_8_data/test_68.csv', 68), ('test_shards_distributed_ordered_8_data/test_69.csv', 69), ('test_shards_distributed_ordered_8_data/test_7.csv', 7), ('test_shards_distributed_ordered_8_data/test_70.csv', 70), ('test_shards_distributed_ordered_8_data/test_71.csv', 71), ('test_shards_distributed_ordered_8_data/test_72.csv', 72), ('test_shards_distributed_ordered_8_data/test_73.csv', 73), ('test_shards_distributed_ordered_8_data/test_74.csv', 74), ('test_shards_distributed_ordered_8_data/test_75.csv', 75), ('test_shards_distributed_ordered_8_data/test_76.csv', 76), ('test_shards_distributed_ordered_8_data/test_77.csv', 77), ('test_shards_distributed_ordered_8_data/test_78.csv', 78), ('test_shards_distributed_ordered_8_data/test_79.csv', 79), ('test_shards_distributed_ordered_8_data/test_8.csv', 8), ('test_shards_distributed_ordered_8_data/test_80.csv', 80), ('test_shards_distributed_ordered_8_data/test_81.csv', 81), ('test_shards_distributed_ordered_8_data/test_82.csv', 82), ('test_shards_distributed_ordered_8_data/test_83.csv', 83), ('test_shards_distributed_ordered_8_data/test_84.csv', 84), ('test_shards_distributed_ordered_8_data/test_85.csv', 85), ('test_shards_distributed_ordered_8_data/test_86.csv', 86), ('test_shards_distributed_ordered_8_data/test_87.csv', 87), ('test_shards_distributed_ordered_8_data/test_88.csv', 88), ('test_shards_distributed_ordered_8_data/test_89.csv', 89), ('test_shards_distributed_ordered_8_data/test_9.csv', 9), ('test_shards_distributed_ordered_8_data/test_90.csv', 90), ('test_shards_distributed_ordered_8_data/test_91.csv', 91), ('test_shards_distributed_ordered_8_data/test_92.csv', 92), ('test_shards_distributed_ordered_8_data/test_93.csv', 93), ('test_shards_distributed_ordered_8_data/test_94.csv', 94), ('test_shards_distributed_ordered_8_data/test_95.csv', 95), ('test_shards_distributed_ordered_8_data/test_96.csv', 96), ('test_shards_distributed_ordered_8_data/test_97.csv', 97), ('test_shards_distributed_ordered_8_data/test_98.csv', 98), ('test_shards_distributed_ordered_8_data/test_99.csv', 99)] ------------------------------ Captured log call ------------------------------- 2026-07-18 08:28:16 [ 514 ] DEBUG : Executing query DROP TABLE IF EXISTS test_shards_distributed_ordered_8 on instance (cluster.py:3606, query) 2026-07-18 08:28:16 [ 514 ] DEBUG : Executing query CREATE TABLE test_shards_distributed_ordered_8 (column1 UInt32, column2 UInt32, column3 UInt32) ENGINE = S3Queue('http://minio1:9001/root/test_shards_distributed_ordered_8_data/', 'minio','minio123', CSV) SETTINGS s3queue_loading_retries=0,after_processing='keep',keeper_path='/clickhouse/test_test_shards_distributed_ordered_8',mode='ordered',s3queue_processing_threads_num=8,s3queue_buckets=2 on instance (cluster.py:3606, query) 2026-07-18 08:28:16 [ 514 ] DEBUG : Executing query DROP TABLE IF EXISTS test_shards_distributed_ordered_8 on instance2 (cluster.py:3606, query) 2026-07-18 08:28:16 [ 514 ] DEBUG : Executing query CREATE TABLE test_shards_distributed_ordered_8 (column1 UInt32, column2 UInt32, column3 UInt32) ENGINE = S3Queue('http://minio1:9001/root/test_shards_distributed_ordered_8_data/', 'minio','minio123', CSV) SETTINGS s3queue_loading_retries=0,after_processing='keep',keeper_path='/clickhouse/test_test_shards_distributed_ordered_8',mode='ordered',s3queue_processing_threads_num=8,s3queue_buckets=2 on instance2 (cluster.py:3606, query) 2026-07-18 08:28:16 [ 514 ] DEBUG : Executing query DROP TABLE IF EXISTS test_shards_distributed_ordered_8_dst; DROP TABLE IF EXISTS test_shards_distributed_ordered_8_dst_mv; CREATE TABLE test_shards_distributed_ordered_8_dst (column1 UInt32, column2 UInt32, column3 UInt32, _path String) ENGINE = MergeTree() ORDER BY column1; CREATE MATERIALIZED VIEW test_shards_distributed_ordered_8_dst_mv TO test_shards_distributed_ordered_8_dst AS SELECT *, _path FROM test_shards_distributed_ordered_8; on instance (cluster.py:3606, query) 2026-07-18 08:28:16 [ 514 ] DEBUG : Executing query DROP TABLE IF EXISTS test_shards_distributed_ordered_8_dst; DROP TABLE IF EXISTS test_shards_distributed_ordered_8_dst_mv; CREATE TABLE test_shards_distributed_ordered_8_dst (column1 UInt32, column2 UInt32, column3 UInt32, _path String) ENGINE = MergeTree() ORDER BY column1; CREATE MATERIALIZED VIEW test_shards_distributed_ordered_8_dst_mv TO test_shards_distributed_ordered_8_dst AS SELECT *, _path FROM test_shards_distributed_ordered_8; on instance2 (cluster.py:3606, query) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_0.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_1.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_10.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_100.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_101.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_102.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_103.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_104.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_105.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_106.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_107.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_108.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_109.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_11.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_110.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_111.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_112.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_113.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_114.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_115.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_116.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_117.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_118.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_119.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_12.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_120.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_121.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_122.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_123.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_124.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_125.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_126.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_127.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_128.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_129.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_13.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_130.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_131.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_132.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_133.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_134.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_135.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_136.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_137.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_138.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_139.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_14.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_140.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_141.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_142.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_143.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_144.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_145.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_146.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_147.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_148.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_149.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_15.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_150.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_151.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_152.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_153.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_154.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_155.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_156.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_157.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_158.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_159.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_16.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_160.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_161.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_162.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_163.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_164.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_165.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_166.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_167.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:16 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_168.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_169.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_17.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_170.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_171.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_172.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_173.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_174.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_175.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_176.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_177.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_178.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_179.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_18.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_180.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_181.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_182.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_183.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_184.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_185.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_186.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_187.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_188.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_189.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_19.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_190.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_191.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_192.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_193.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_194.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_195.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_196.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_197.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_198.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_199.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_2.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_20.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_200.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_201.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_202.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_203.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_204.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_205.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_206.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_207.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_208.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_209.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_21.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_210.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_211.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_212.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_213.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_214.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_215.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_216.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_217.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_218.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_219.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_22.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_220.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_221.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_222.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_223.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_224.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_225.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_226.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_227.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_228.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_229.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_23.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_230.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_231.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_232.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_233.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_234.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_235.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_236.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_237.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_238.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_239.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_24.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_240.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_241.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_242.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_243.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_244.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_245.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_246.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_247.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_248.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_249.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_25.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_250.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_251.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_252.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_253.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_254.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_255.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_256.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_257.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_258.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_259.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_26.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_260.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_261.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_262.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_263.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_264.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_265.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_266.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_267.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_268.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_269.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_27.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_270.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_271.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_272.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_273.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_274.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_275.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_276.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_277.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_278.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_279.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_28.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_280.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_281.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_282.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_283.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_284.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_285.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_286.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_287.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_288.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_289.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_29.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_290.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_291.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_292.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_293.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_294.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_295.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_296.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_297.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_298.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_299.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_3.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_30.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_31.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_32.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_33.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_34.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_35.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_36.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_37.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_38.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_39.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_4.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_40.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_41.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_42.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_43.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_44.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_45.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_46.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_47.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_48.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_49.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_5.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_50.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_51.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_52.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_53.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_54.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_55.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_56.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_57.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_58.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_59.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_6.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_60.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_61.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_62.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_63.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_64.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_65.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_66.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_67.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_68.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_69.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_7.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_70.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_71.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_72.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_73.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_74.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_75.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_76.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_77.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_78.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_79.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_8.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_80.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_81.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_82.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_83.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_84.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_85.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_86.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_87.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_88.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_89.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_9.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_90.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_91.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_92.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_93.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_94.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_95.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_96.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_97.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_98.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_99.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:17 [ 514 ] INFO : Running query 'SELECT count() FROM test_shards_distributed_ordered_8_dst'... (test.py:160, run_query) 2026-07-18 08:28:17 [ 514 ] DEBUG : Executing query SELECT count() FROM test_shards_distributed_ordered_8_dst on instance (cluster.py:3606, query) 2026-07-18 08:28:17 [ 514 ] INFO : Query finished (test.py:162, run_query) 2026-07-18 08:28:17 [ 514 ] INFO : Running query 'SELECT count() FROM test_shards_distributed_ordered_8_dst'... (test.py:160, run_query) 2026-07-18 08:28:17 [ 514 ] DEBUG : Executing query SELECT count() FROM test_shards_distributed_ordered_8_dst on instance2 (cluster.py:3606, query) 2026-07-18 08:28:17 [ 514 ] INFO : Query finished (test.py:162, run_query) 2026-07-18 08:28:18 [ 514 ] INFO : Running query 'SELECT count() FROM test_shards_distributed_ordered_8_dst'... (test.py:160, run_query) 2026-07-18 08:28:18 [ 514 ] DEBUG : Executing query SELECT count() FROM test_shards_distributed_ordered_8_dst on instance (cluster.py:3606, query) 2026-07-18 08:28:18 [ 514 ] INFO : Query finished (test.py:162, run_query) 2026-07-18 08:28:18 [ 514 ] INFO : Running query 'SELECT count() FROM test_shards_distributed_ordered_8_dst'... (test.py:160, run_query) 2026-07-18 08:28:18 [ 514 ] DEBUG : Executing query SELECT count() FROM test_shards_distributed_ordered_8_dst on instance2 (cluster.py:3606, query) 2026-07-18 08:28:18 [ 514 ] INFO : Query finished (test.py:162, run_query) 2026-07-18 08:28:19 [ 514 ] INFO : Running query 'SELECT count() FROM test_shards_distributed_ordered_8_dst'... (test.py:160, run_query) 2026-07-18 08:28:19 [ 514 ] DEBUG : Executing query SELECT count() FROM test_shards_distributed_ordered_8_dst on instance (cluster.py:3606, query) 2026-07-18 08:28:19 [ 514 ] INFO : Query finished (test.py:162, run_query) 2026-07-18 08:28:19 [ 514 ] INFO : Running query 'SELECT count() FROM test_shards_distributed_ordered_8_dst'... (test.py:160, run_query) 2026-07-18 08:28:19 [ 514 ] DEBUG : Executing query SELECT count() FROM test_shards_distributed_ordered_8_dst on instance2 (cluster.py:3606, query) 2026-07-18 08:28:19 [ 514 ] INFO : Query finished (test.py:162, run_query) 2026-07-18 08:28:20 [ 514 ] INFO : Running query 'SELECT count() FROM test_shards_distributed_ordered_8_dst'... (test.py:160, run_query) 2026-07-18 08:28:20 [ 514 ] DEBUG : Executing query SELECT count() FROM test_shards_distributed_ordered_8_dst on instance (cluster.py:3606, query) 2026-07-18 08:28:20 [ 514 ] INFO : Query finished (test.py:162, run_query) 2026-07-18 08:28:20 [ 514 ] INFO : Running query 'SELECT count() FROM test_shards_distributed_ordered_8_dst'... (test.py:160, run_query) 2026-07-18 08:28:20 [ 514 ] DEBUG : Executing query SELECT count() FROM test_shards_distributed_ordered_8_dst on instance2 (cluster.py:3606, query) 2026-07-18 08:28:21 [ 514 ] INFO : Query finished (test.py:162, run_query) 2026-07-18 08:28:22 [ 514 ] INFO : Running query 'SELECT count() FROM test_shards_distributed_ordered_8_dst'... (test.py:160, run_query) 2026-07-18 08:28:22 [ 514 ] DEBUG : Executing query SELECT count() FROM test_shards_distributed_ordered_8_dst on instance (cluster.py:3606, query) 2026-07-18 08:28:22 [ 514 ] INFO : Query finished (test.py:162, run_query) 2026-07-18 08:28:22 [ 514 ] INFO : Running query 'SELECT count() FROM test_shards_distributed_ordered_8_dst'... (test.py:160, run_query) 2026-07-18 08:28:22 [ 514 ] DEBUG : Executing query SELECT count() FROM test_shards_distributed_ordered_8_dst on instance2 (cluster.py:3606, query) 2026-07-18 08:28:22 [ 514 ] INFO : Query finished (test.py:162, run_query) 2026-07-18 08:28:23 [ 514 ] INFO : Running query 'SELECT count() FROM test_shards_distributed_ordered_8_dst'... (test.py:160, run_query) 2026-07-18 08:28:23 [ 514 ] DEBUG : Executing query SELECT count() FROM test_shards_distributed_ordered_8_dst on instance (cluster.py:3606, query) 2026-07-18 08:28:23 [ 514 ] INFO : Query finished (test.py:162, run_query) 2026-07-18 08:28:23 [ 514 ] INFO : Running query 'SELECT count() FROM test_shards_distributed_ordered_8_dst'... (test.py:160, run_query) 2026-07-18 08:28:23 [ 514 ] DEBUG : Executing query SELECT count() FROM test_shards_distributed_ordered_8_dst on instance2 (cluster.py:3606, query) 2026-07-18 08:28:23 [ 514 ] INFO : Query finished (test.py:162, run_query) 2026-07-18 08:28:23 [ 514 ] INFO : Running query 'SELECT count() FROM test_shards_distributed_ordered_8_dst'... (test.py:160, run_query) 2026-07-18 08:28:23 [ 514 ] DEBUG : Executing query SELECT count() FROM test_shards_distributed_ordered_8_dst on instance (cluster.py:3606, query) 2026-07-18 08:28:23 [ 514 ] INFO : Query finished (test.py:162, run_query) 2026-07-18 08:28:23 [ 514 ] INFO : Running query 'SELECT count() FROM test_shards_distributed_ordered_8_dst'... (test.py:160, run_query) 2026-07-18 08:28:23 [ 514 ] DEBUG : Executing query SELECT count() FROM test_shards_distributed_ordered_8_dst on instance2 (cluster.py:3606, query) 2026-07-18 08:28:23 [ 514 ] INFO : Query finished (test.py:162, run_query) 2026-07-18 08:28:23 [ 514 ] INFO : Running query 'SELECT column1, column2, column3 FROM test_shards_distributed_ordered_8_dst'... (test.py:160, run_query) 2026-07-18 08:28:23 [ 514 ] DEBUG : Executing query SELECT column1, column2, column3 FROM test_shards_distributed_ordered_8_dst on instance (cluster.py:3606, query) 2026-07-18 08:28:23 [ 514 ] INFO : Query finished (test.py:162, run_query) 2026-07-18 08:28:23 [ 514 ] INFO : Running query 'SELECT column1, column2, column3 FROM test_shards_distributed_ordered_8_dst'... (test.py:160, run_query) 2026-07-18 08:28:23 [ 514 ] DEBUG : Executing query SELECT column1, column2, column3 FROM test_shards_distributed_ordered_8_dst on instance2 (cluster.py:3606, query) 2026-07-18 08:28:23 [ 514 ] INFO : Query finished (test.py:162, run_query) 2026-07-18 08:28:23 [ 514 ] DEBUG : res1 size: 15000, res2 size: 0, total_rows: 15000 (test.py:1459, test_shards_distributed) 2026-07-18 08:28:23 [ 514 ] DEBUG : Executing query select splitByChar('/', file_name)[-1] as file from system.s3queue where zookeeper_path ilike '%test_shards_distributed_ordered_8%' and status = 'Processed' and rows_processed > 0 order by file on instance (cluster.py:3606, query) 2026-07-18 08:28:23 [ 514 ] DEBUG : Processed files by node 1: 300/300 (test.py:1384, print_debug_info) 2026-07-18 08:28:23 [ 514 ] DEBUG : Executing query select splitByChar('/', file_name)[-1] as file from system.s3queue where zookeeper_path ilike '%test_shards_distributed_ordered_8%' and status = 'Processed' and rows_processed > 0 order by file on instance2 (cluster.py:3606, query) 2026-07-18 08:28:23 [ 514 ] DEBUG : Processed files by node 2: 1/300 (test.py:1396, print_debug_info) 2026-07-18 08:28:23 [ 514 ] INFO : Running query 'SELECT count() FROM test_shards_distributed_ordered_8_dst'... (test.py:160, run_query) 2026-07-18 08:28:23 [ 514 ] DEBUG : Executing query SELECT count() FROM test_shards_distributed_ordered_8_dst on instance (cluster.py:3606, query) 2026-07-18 08:28:23 [ 514 ] INFO : Query finished (test.py:162, run_query) 2026-07-18 08:28:23 [ 514 ] INFO : Running query 'SELECT count() FROM test_shards_distributed_ordered_8_dst'... (test.py:160, run_query) 2026-07-18 08:28:23 [ 514 ] DEBUG : Executing query SELECT count() FROM test_shards_distributed_ordered_8_dst on instance2 (cluster.py:3606, query) 2026-07-18 08:28:23 [ 514 ] INFO : Query finished (test.py:162, run_query) 2026-07-18 08:28:23 [ 514 ] DEBUG : Processed rows: 15000/15000 (test.py:1401, print_debug_info) 2026-07-18 08:28:23 [ 514 ] DEBUG : Executing query select concat('test_', toString(number), '.csv') as file from numbers(300) where file not in (select splitByChar('/', file_name)[-1] from clusterAllReplicas(default, system.s3queue) where zookeeper_path ilike '%test_shards_distributed_ordered_8%' and status = 'Processed' and rows_processed > 0) on instance (cluster.py:3606, query) 2026-07-18 08:28:23 [ 514 ] DEBUG : Unprocessed files: (test.py:1410, print_debug_info) 2026-07-18 08:28:23 [ 514 ] DEBUG : Executing query select splitByChar('/', file_name)[-1] from system.s3queue where zookeeper_path ilike '%test_shards_distributed_ordered_8%' and status = 'Processed' and rows_processed > 0 on instance (cluster.py:3606, query) 2026-07-18 08:28:23 [ 514 ] DEBUG : Executing query select splitByChar('/', file_name)[-1] from system.s3queue where zookeeper_path ilike '%test_shards_distributed_ordered_8%' and status = 'Processed' and rows_processed > 0 on instance2 (cluster.py:3606, query) 2026-07-18 08:28:24 [ 514 ] DEBUG : Intersecting files: [] (test.py:1436, print_debug_info) ---------------------------- Captured log teardown ----------------------------- 2026-07-18 08:28:24 [ 514 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/.env', '--project-name', 'rootteststorages3queue', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance2/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/old_instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance_too_many_parts/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance_24.5/docker-compose.yml', 'stop', '--timeout', '20'] (cluster.py:113, run_and_check) 2026-07-18 08:28:45 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_instance2_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:45 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_instance_24.5_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:45 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_old_instance_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:45 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_instance_too_many_parts_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:45 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_instance_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:45 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_azurite1_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:45 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_resolver_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:45 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_minio1_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:45 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_proxy2_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:45 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_proxy1_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:45 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_zoo1_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:45 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_zoo2_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:45 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_zoo3_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:45 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_instance_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:45 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_old_instance_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:45 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_instance_24.5_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:45 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_instance_too_many_parts_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:45 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_instance2_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:45 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_minio1_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:45 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_zoo2_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:45 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_azurite1_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:45 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_zoo1_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:45 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_zoo3_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:45 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_resolver_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:45 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_proxy1_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:45 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_proxy2_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:45 [ 514 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:113, run_and_check) 2026-07-18 08:28:45 [ 514 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance2/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance2/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:113, run_and_check) 2026-07-18 08:28:45 [ 514 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/old_instance/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/old_instance/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:113, run_and_check) 2026-07-18 08:28:45 [ 514 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance_too_many_parts/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance_too_many_parts/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:113, run_and_check) 2026-07-18 08:28:45 [ 514 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance_24.5/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance_24.5/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:113, run_and_check) 2026-07-18 08:28:45 [ 514 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/.env', '--project-name', 'rootteststorages3queue', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance2/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/old_instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance_too_many_parts/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_1/instance_24.5/docker-compose.yml', 'down', '--volumes'] (cluster.py:113, run_and_check) 2026-07-18 08:28:46 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_instance2_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:46 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_instance_24.5_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:46 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_old_instance_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:46 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_instance_too_many_parts_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:46 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_instance_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:46 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_azurite1_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:46 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_resolver_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:46 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_minio1_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:46 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_proxy2_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:46 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_proxy1_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:46 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_zoo1_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:46 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_zoo2_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:46 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_zoo3_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:46 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_proxy1_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:46 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_instance2_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:46 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_instance_too_many_parts_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:46 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_azurite1_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:46 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_zoo3_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:46 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_instance_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:46 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_proxy2_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:46 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_instance_24.5_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:46 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_old_instance_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:46 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_zoo2_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:46 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_zoo1_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:46 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_resolver_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:46 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_minio1_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:46 [ 514 ] DEBUG : Stderr:Removing network rootteststorages3queue_default (cluster.py:123, run_and_check) 2026-07-18 08:28:46 [ 514 ] DEBUG : Stderr:Removing volume rootteststorages3queue_data1-1 (cluster.py:123, run_and_check) 2026-07-18 08:28:46 [ 514 ] DEBUG : Cleanup called (cluster.py:876, cleanup) 2026-07-18 08:28:46 [ 514 ] DEBUG : Docker networks for project rootteststorages3queue are NETWORK ID NAME DRIVER SCOPE (cluster.py:855, print_all_docker_pieces) 2026-07-18 08:28:46 [ 514 ] DEBUG : Docker containers for project rootteststorages3queue are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:863, print_all_docker_pieces) 2026-07-18 08:28:46 [ 514 ] DEBUG : Docker volumes for project rootteststorages3queue are DRIVER VOLUME NAME (cluster.py:871, print_all_docker_pieces) 2026-07-18 08:28:46 [ 514 ] DEBUG : Command:docker container list --all --filter name='^/rootteststorages3queue_.*_1$' --format '{{.ID}}:{{.Names}}' (cluster.py:113, run_and_check) 2026-07-18 08:28:46 [ 514 ] DEBUG : Unstopped containers: {} (cluster.py:890, cleanup) 2026-07-18 08:28:46 [ 514 ] DEBUG : No running containers for project: rootteststorages3queue (cluster.py:904, cleanup) 2026-07-18 08:28:46 [ 514 ] DEBUG : Trying to prune unused networks... (cluster.py:910, cleanup) 2026-07-18 08:28:46 [ 514 ] DEBUG : Trying to prune unused volumes... (cluster.py:937, cleanup) 2026-07-18 08:28:46 [ 514 ] DEBUG : Command:['docker volume ls | wc -l'] (cluster.py:113, run_and_check) 2026-07-18 08:28:46 [ 514 ] DEBUG : Stdout:1 (cluster.py:121, run_and_check) ============================== slowest durations =============================== 22.09s teardown test_storage_s3_queue/test.py::test_shards_distributed[ordered-8] 14.56s setup test_storage_s3_queue/test.py::test_shards_distributed[ordered-8] 7.48s call test_storage_s3_queue/test.py::test_shards_distributed[ordered-8] =========================== short test summary info ============================ FAILED test_storage_s3_queue/test.py::test_shards_distributed[ordered-8] - as... ============================== 1 failed in 44.20s ============================== Traceback (most recent call last): File "/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration/./runner", line 445, in subprocess.check_call(cmd, shell=True) File "/usr/lib/python3.10/subprocess.py", line 369, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command 'docker run --rm --name clickhouse_integration_tests_kxmknw --privileged --dns-search='.' --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-odbc-bridge:/clickhouse-odbc-bridge --volume=/home/ubuntu/_work/_temp/test/build/clickhouse:/clickhouse --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-library-bridge:/clickhouse-library-bridge --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/programs/server:/clickhouse-config --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration:/ClickHouse/tests/integration --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/backupview:/ClickHouse/utils/backupview --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/grpc-client/pb2:/ClickHouse/utils/grpc-client/pb2 --volume=/run:/run/host:ro --volume=clickhouse_integration_tests_volume:/var/lib/docker -e DOCKER_DOTNET_CLIENT_TAG=11de0b29a15d -e DOCKER_HELPER_TAG=2cffe1eae894 -e DOCKER_BASE_TAG=b74d853ebe76 -e DOCKER_KERBEROS_KDC_TAG=9391ecdee8d7 -e DOCKER_MYSQL_GOLANG_CLIENT_TAG=9bec2a638e6e -e DOCKER_MYSQL_JAVA_CLIENT_TAG=766bff31cfe4 -e DOCKER_MYSQL_JS_CLIENT_TAG=41ba7c2ec2a1 -e DOCKER_MYSQL_PHP_CLIENT_TAG=88be89c1e3b6 -e DOCKER_NGINX_DAV_TAG=b55ac9cd7519 -e DOCKER_POSTGRESQL_JAVA_CLIENT_TAG=a4eff5c7f4d6 -e DOCKER_PYTHON_BOTTLE_TAG=caad4729259e -e DOCKER_CLIENT_TIMEOUT=300 -e COMPOSE_HTTP_TIMEOUT=600 -e PYTHONUNBUFFERED=1 -e PYTEST_ADDOPTS=" -rfEps --run-id=1 --color=no --durations=0 'test_storage_s3_queue/test.py::test_shards_distributed[ordered-8]' -vvv" altinityinfra/integration-tests-runner:cf95077f4609 ' returned non-zero exit status 1.